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Abstract 


The  purpose  of  this  study  was  to  determine  if  survey  medium  (paper  versus 
computer)  affected  responses  and  response  rates  in  Air  Force  personnel.  The  study 
compared  responses  and  response  rates  from  900  randomly  selected  Air  Force  active- 
duty  members  using  a  paper-based  survey,  a  computer-based  survey,  and  a  more  complex 
computer-based  survey.  The  first  computer-based  survey  minimized  the  differences 
between  itself  and  the  paper-based  survey  to  more  accurately  quantify  any  bias  due  solely 
to  the  computer  medium.  The  more  complex  survey  served  to  maximize  differences 
between  itself  and  the  other  computer-based  survey  to  more  accurately  quantify  any  bias 
due  to  programmatic  complexity.  In  addition,  responses  from  groups  stratified  on  gender 
(men  and  women)  and  military  commission  (officers  and  enlisted)  were  compared 
between  the  three  survey  types.  The  results  showed  that  no  statistically  significant 
differences  could  be  detected  between  the  paper  and  computer  surveys  overall  and  for 
men,  women,  officer,  and  enlisted  personnel.  In  the  context  of  non-sensitive, 
organizational  research,  paper  and  computer  surveys  can  be  considered  equivalent 
research  mediums  with  regard  to  reliability  and  validity. 
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COMPARATIVE  ANALYSIS  OF 


TRADITIONAL  VERSUS  COMPUTER-BASED  SURVEY 
INSTRUMENT  RESPONSE 

I.  Introduction 

Survey  research  is  considered  the  most  popular  and  widely  used  method  to 
conduct  social  science  research  today  (Babbie,  1990).  Not  limited  to  academia,  survey 
research  is  conducted  by  a  wide  variety  of  organizations  from  small  businesses  and 
marketing  companies  to  government  agencies.  Survey  instruments  are  often  used 
because  of  their  ability  to  gather  information  not  possible  to  be  gathered  by  any  other 
method.  Because  of  the  widespread  use  of  surveys,  inevitable  problems  arise  that  cause 
criticism  to  be  leveled  against  the  findings  of  many  surveys.  In  particular,  poorly  written 
surveys,  poor  survey  techniques,  survey  overuse,  and  survey  misuse  all  have  lead  to  the 
outright  rejection  of  survey  research  by  many  groups  and  individuals  (Babbie,  1990; 
Hugnagel  &  Conca,  1994).  To  combat  this  criticism,  increased  attention  must  be  given  to 
survey  instrument  development  and  administration  to  assure  that  the  survey’s 
“significance  cannot  be  doubted”  (Parten,  1950:1),  in  terms  of  reliability  and  validity. 
This  thesis  focuses  on  one  aspect  of  modem  survey  research  that  has  generated 
controversy  stemming  from  early  studies  in  Computer  Assisted  Testing  (CAT).  This 
aspect  concerns  the  introduction  of  the  computer  as  a  medium  to  administer  surveys  and 
the  effects  the  computer  may  have  on  human  response. 
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Background 

Traditionally,  survey  instruments,  which  encompass  the  use  of  questionnaires, 
polls,  and  psychological  tests,  have  been  administered  through  a  paper-based  medium 
because  of  the  availability,  low  cost,  and  acceptance  of  this  most  common  of 
informational  mediums  (Rosenfeld  &  Booth-Kewely,  1993).  The  reliability  levels, 
regarding  whether  questions  produce  consistent  measures  in  similar  situations,  and 
validity  levels,  regarding  whether  answers  correspond  to  the  measurement  under  study,  of 
gathering  information  using  a  paper  medium  are  often  statistically  grounded  in  the 
research  that  uses  them  (Fowler,  1988).  As  new  technologies  emerge,  researchers  must 
empirically  study  changes  in  reliabilities  and  validities  from  gathering  information 
through  traditional  means  to  gathering  information  through  those  new  technologies.  For 
example,  researchers  compared  survey  results  between  using  paper  and  the  telephone, 
finding  significant  differences  (Hochstim,  1967;  Rogers,  1976).  Although  intuition 
suggests  that  different  media  might  produce  different  responses,  research  must  document 
and  build  a  body  of  evidence  to  support  the  equivalency  or  nonequivalence  of  the  change 
in  survey  mediums. 

Recently,  the  accessibility,  ease-of-use,  and  processing  power  of  modem 
computers  have  enabled  organizations,  including  many  in  the  Air  Force  and  US 
government  in  general,  to  rely  on  computer-based  surveys,  such  as  the  recent  Internet- 
based  Quality  of  Life  Survey,  much  more  than  a  few  years  ago.  It  is  now  quite  simple  to 
create  a  computer-based  survey  and  quickly  administer  it  to  a  number  of  people 
simultaneously  on  stand-alone  computers,  Local  Area  Networks  (LANs),  or  over  the 
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Internet.  However,  many  organizations  do  not  question  whether  validity  and  reliability 
are  affected  by  the  new  method  of  administration,  which  presents  a  significant  problem. 

Problem  Statement 

Using  computers  to  administer  surveys  may  introduce  the  potential  for  errors  or 
bias  that  was  previously  unaccounted  for  by  researchers.  That  potential  must  be 
empirically  assessed.  In  this  context,  the  computer-based  survey  medium  encompasses 
the  visual  presentation,  physical  manipulation,  user  and  computer  processes,  and 
constraints  and  limitations  of  using  the  computer  system.  Humans  are  traditionally 
accustomed  to  a  paper-based  medium  for  practically  everything  from  reading  and  writing 
to  tests  and  questionnaires  (Rosenfeld  et  al.,  1993).  The  introduction  of  the  computer  to 
conduct  studies  once  performed  using  paper  and  pencil  has  stirred  much  debate  about  the 
differences  in  response  people  have  from  one  form  to  the  other.  One  study  comparing 
letters  written  on  both  paper  and  computer  found  that  the  computer-based  writings  were 
“poorer  in  content  quality  and  total  quality”  (Haas,  1989:149).  Other  studies  comparing 
reading  comprehension  in  both  the  paper  and  computer  mediums  found  that  learning  was 
enhanced  through  the  computer  medium  (Higgins  &  Hess,  1999;  Homey  &  Anderson- 
Inman,  1999).  However,  Regan  (2000:18)  suggests  that  electronic  books  do  not  allow  for 
a  “pleasant  reading  experience.”  How  then  does  the  computer  medium  change  how 
people  respond  to  surveys  compared  with  the  popular  and  traditional  paper  medium? 

Although  the  future  may  prove  the  computer  to  be  the  most  popular  medium 
rather  than  paper,  the  computer,  currently,  adds  an  element  of  unfamiliarity  or  difficulty 
for  many  people  (Comley,  2000).  Because  of  this,  some  people  or  groups  may  not  want 
to  answer  a  computer-based  survey.  This  unwillingness  or  inability  to  answer  can 
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influence  response  rates  in  the  form  of  non-response  bias  (Kiesler,  1986).  At  the  other 
end  of  the  spectrum,  some  individuals  and  groups  may  feel  more  comfortable  answering 
a  computer-based  survey  and  may  reduce  the  non-response  rate.  Some  people  or 
subgroups  may  respond  to  computer-based  survey  questions  differently  than  other  people 
or  subgroups,  intentionally  or  unintentionally,  in  the  form  of  response  bias  (Paulhus, 
1991).  Response  bias  is  any  number  of  internal  and  external  factors  that  affect  how 
people  answer  questions  on  a  survey.  These  two  concepts  of  bias,  which  are  the 
dependent  variables  in  this  study,  will  be  thoroughly  explored  in  the  literature  review. 
Research  Questions  (RQ) 

This  study  focuses  on  three  objectives.  The  first  and  primary  objective  of  this 
research  is  to  determine  if  computer-based  surveys  are  equivalent  to  analogous  paper- 
based  surveys.  Two  surveys  can  be  considered  equivalent  if  “they  produce  equal  mean 
scores,  identical  distribution  and  ranking  of  scores,  and  correlate  to  the  exact  same  degree 
with  scores  on  any  other  variable”  (Ghiselli,  1964:227;  Honaker,  1988:562).  This 
research  will  determine  if  the  computer  medium  introduces  significant  bias  that  causes 
the  response  rate  or  mean  scores  to  be  significantly  different  because  of  the  effect  of  the 
survey  instrument  medium.  Additionally,  this  research  will  attempt  to  quantify  any  bias 
that  is  due  solely  to  the  fact  that  the  survey  interface  is  a  computer  due  to  differences  in 
personality,  apprehension,  difficulty,  interference  from  the  computer,  or  a  combination  of 
all  these  internal  and  external  factors. 

The  second  objective  is  to  determine  if  format  and  complexity  differences 
between  computer-based  surveys  introduce  additional  bias.  Cizek  (1994)  and  others 
found  that  even  the  smallest  changes  to  tests  could  produce  statistically  different 
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responses.  Altering  the  position  of  correct  responses  on  a  multiple-choice  exam  was 
found  to  affect  user  responses  to  equated  examinations.  Webster  et  al.  (1996:568) 
suggest  that  format  and  complexity  differences  affect  computer  users  through  an  inability 
to  backtrack,  increased  attention  on  items  when  individually  presented,  immediate  and 
automatic  feedback,  initial  cursor  positioning,  and  “test-taker  feelings  of  a  lack  of  control 
if  answers  cannot  be  changed  during  computer  testing.” 

The  third  objective  is  to  determine  if  particular  groups  stratified  on  gender  (men 
versus  women)  and/or  military  commission  (officer  versus  enlisted)  are  affected 
differently  by  survey  mediums.  Previous  research  into  gender  and  computer  technology 
has  shown  differences  in  the  way  men  and  women  perceive  computer  technology 
(Venkatesh  &  Morris,  2000;  Gefen  &  Straub,  1997).  Although  no  studies  in  the  literature 
review  directly  reviewed  differences  between  perceptions  of  computer  technology 
between  officers  and  enlisted  personnel,  this  study  attempts  make  inferences  about  these 
groups  based  on  education  level  taking  into  account  other  differences  such  as  leadership 
training  and  military  social  class  differences.  Rogers  (1995:269)  found  that  people  with 
“more  years  of  formal  education”  adopt  innovations,  to  include  information  technology 
(IT),  earlier  than  those  with  fewer  years  of  formal  education.  In  addition,  Rogers 
(1995:269)  says  that  those  with  “higher  social  status”  and  “greater  degree  of  social 
mobility”  will  use  and  accept  IT  much  sooner  than  those  lacking  these  characteristics. 
These  attributes  can  clearly  be  associated  with  members  of  the  officer  force  at  a  higher 
degree  than  those  members  of  the  enlisted  force,  suggesting  that  officers  should  be  more 
accepting  of  IT  and  web-based  innovations  than  enlisted  personnel. 
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Based  on  the  stated  research  objectives,  three  research  questions  (RQ)  naturally 
form,  whose  associated  hypotheses  will  be  proposed  during  the  discussion  in  Chapter  2: 

RQ1)  Are  computer-based  and  paper-based  survey  instruments  equivalent? 

RQ2)  Do  complex  computer-based  surveys  introduce  significant  bias  into 

survey  responses? 

RQ3)  Are  computer-based  and  paper-based  survey  responses  or  response 

rates  affected  by  a  person’s  gender  or  military  commission? 

Significance  of  this  Study 

Academic  and  vocational  researchers  cannot  assume  that  the  constructs  measured 
with  paper-based  surveys,  polls,  questionnaires,  or  psychological  tests  can  be 
equivalently  measured  with  a  similar  computer-based  version  of  the  paper-based 
instrument.  While  a  significant  body  of  research  has  been  conducted  regarding  the 
equivalence  of  different  methods  of  survey  and  test  administration,  results  have  been 
mixed  and  sometimes  contradictory  (Booth-Kewley  et  al.,  1992:562).  Additionally,  a 
significant  aspect  of  survey  equivalence,  the  measurement  of  non-response,  has  not  been 
adequately  addressed  in  most  studies.  This  study  will  determine,  more  accurately,  non¬ 
response  and  response  bias  produced  when  Air  Force  members  have  a  choice  to  complete 
a  survey  and  self-report  on  their  own  time,  on  their  own  terms,  and  without  appreciable 
pressure  from  the  researcher. 

Results  from  this  study  will  help  confirm  the  use  of  computer-based  surveys  or 
disconfirm  their  use  under  certain  circumstances.  If  this  research  shows  that  there  is  a 
difference  in  how  people  voluntarily  answer  computer  and  paper-based  surveys, 
organizations  that  create  or  analyze  survey  data  can  make  changes  to  account  for  these 
effects.  For  example,  if  this  research  shows  that  officers  tend  to  over-report  positive 
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feelings  on  computer  surveys,  future  computer  surveys  may  incorporate  questions  to 
control  for  this  tendency.  However,  if  the  research  shows  that  there  is  no  significant 
difference  between  the  two  methods,  then  validity  has  been  strengthened  for  the  use  of 
computer-based  survey  instruments  without  prejudice. 

With  studies  like  this  one,  researchers  can  say  with  confidence  that  the  results 
from  their  computer-based  surveys  are  valid  compared  with  equivalent  results  from 
previous  paper-and-pencil  surveys.  Without  determining  this  validity  and  equivalence, 
researchers  may  possibly  draw  incorrect  conclusions  from  unknowingly  biased  data 
causing  unfounded  policies  to  be  set  by  those  in  command. 

Scope  and  Assumptions 

The  scope  of  this  study  is  all  Air  Force  active  duty  personnel  who  self-report 
questionnaire  responses  in  a  completely  voluntary  environment.  The  results  do  not 
necessarily  apply  to  all  surveys  or  tests  unless  they  are  conducted  under  similar 
circumstances.  Since  the  surveys  for  this  study  are  highly  structured,  closed-ended,  and 
voluntarily  completed  without  pressure,  the  findings  may  not  generalize  to  loosely 
structured  or  open-ended  surveys  administered  under  controlled  environments. 

A  major  assumption  in  this  study  is  that  the  paper  medium  is  seen  as  an 
insignificant  and  unobtrusive  medium.  Every  survey  introduces  some  type  of  bias  (Berg, 
1954;  DeLamater,  1982).  This  is  a  major  problem  found  in  social  science  research  today, 
and  researchers  do  their  best  to  control  for  it.  This  research  will  only  find  any  bias  due  to 
the  change  in  medium  and  format  that  may  or  may  not  affect  survey  reliability  or  validity. 
The  nature  of  this  research  is  exploratory;  at  this  point,  there  is  no  way  of  telling  whether 
the  computer  will  have  a  negative  or  positive  influence  on  responses. 
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Thesis  Structure 


In  the  following  chapter,  justification  for  investigation  into  the  various  aspects  of 
differing  survey  administration  and  formats  will  be  explored.  Special  emphasis  will  be 
placed  on  equivalency  theories,  theories  of  response  effects  and  bias,  survey  complexity 
and  format  differences,  gender  differences  and  military  commission  differences  based  on 
educational,  training  and  social  differences  in  relation  to  IT.  Additionally,  a  thorough 
examination  of  the  previous  research  into  the  subject  of  computer  versus  paper- and- 
pencil  equivalency  will  be  explored.  Chapter  3  will  explain  the  research  method,  the 
different  surveys  used  to  gather  data  from  a  sample  of  Air  Force  personnel,  and  the 
explicit  procedures  used  to  conduct  the  survey.  In  addition,  a  review  of  the  analysis 
techniques  to  be  used  on  the  resultant  data  will  be  examined.  The  final  chapters  will 
analyze  the  actual  data  from  the  survey  and  provide  interpretation  and  discussion  of  the 
results.  Finally,  suggestions  will  be  proposed  for  further  research  that  could  not  be 
addressed  in  the  present  study. 
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II.  Literature  Review 


Chapter  Overview 

This  chapter  consists  of  a  literature  review  of  the  major  constructs,  concepts, 
ideas,  and  research  contributions  of  survey  methodologies  that  have  brought  about  the 
research  questions  and  hypotheses  proposed  in  this  current  study.  Because  this  study  is  a 
comparative  analysis  of  possibly  different  responses  elicited  by  a  computer  versus  a 
paper  survey  medium,  the  concept  of  survey  equivalency  (Honaker  et.al.,  1988)  will  be 
reviewed  as  it  pertains  to  types  of  survey  instruments. 

Next,  the  sources  of  survey  error  and  the  constructs  of  response  and  non-response 
bias  will  be  thoroughly  decomposed,  shedding  light  on  the  varying  degrees  of  overall 
error  that  can  occur.  As  a  further  explanation  of  response  and  non-response  bias,  the 
theories  that  differences  may  be  found  based  on  format,  complexity,  gender,  and 
education  will  be  presented. 

Finally,  this  chapter  will  review  previous  survey  research  encapsulating  the  time 
period  when  most  of  the  discussions  on  the  equivalence  of  computers  and  paper  have 
surfaced.  This  review  will  explicate  the  weaknesses  and  strengths  in  previous  research, 
which  led  to  the  questions  in  this  study  that  require  exploration  and  resolution. 
Equivalency  and  Classical  Test  Theory 

A  recurring  theme  in  the  study  of  the  computer  in  the  administration  of  test  and 
survey  instruments  has  been  the  concept  of  equivalency.  Researchers  are  primarily 
concerned  with  the  equivalency  of  the  computer-based  format  compared  to  the  traditional 
paper-based  format  above  all  other  factors  (e.g.,  Hofer  &  Green,  1985;  Kiesler  &  Sproull, 
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1986;  Honaker,  1988;  Lautenschlager  &  Flaherty,  1990;  Booth-Kewley,  Edwards  & 
Rosenfeld  et  al.,  1991;  Webster  &  Compeau,  1996;  and  others). 

Under  Classical  Test  Theory,  a  computer-based  and  paper-based  survey  can  be 
considered  equivalent  if  “they  produce  equal  mean  scores,  identical  distribution  and 
ranking  of  scores,  and  correlate  to  the  exact  same  degree  with  scores  on  any  other 
variable”  (Ghiselli,  1964:227;  Honaker  et  al.,  1988:562).  This  study  suggests  that 
response  rate  should  also  be  identical  for  the  two  forms  to  be  considered  equivalent  in  a 
self-report  experimental  design.  If  the  response  rate  (number  of  surveys  retumed/number 
of  surveys  sent  to  sample)  is  statistically  different  between  two  methods  of 
administration,  some  additional  and  possibly  unanticipated  factor  is  causing  the 
difference.  Therefore,  equivalency  of  the  paper  and  computer  formats  of  the  same  survey 
cannot  be  automatically  solely  on  Classical  Test  Theory’s  definition.  Before  reviewing 
the  concept  of  response  rates  as  a  factor  of  equivalency,  it  is  important  to  discuss 
Classical  Test  Theory  and  each  “criteria  for  equivalency”  (Honaker,  1988:561).  The 
three  criteria  for  equivalency  are  psychometric  equivalency,  experiential  equivalency, 
and  relativity  of  equivalency  (Honaker,  1988:562). 

Psychometric  equivalency  has  already  been  introduced  as  equivalency  based  on 
two  formats  producing  identical  mean  scores,  distribution,  ranking,  and  correlations  with 
other  variables.  Honaker  (1988)  states  that  if  these  criteria  are  met,  the  validity  can  be 
generalized  from  one  form  to  the  other.  However,  Honaker  (1988)  states  that  if  the 
criteria  are  not  met,  “it  is  likely  that  separate  constructs  are  being  measured,”  and  validity 
data  cannot  be  generalized  from  one  form  to  the  other  form. 
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Another  concept  of  equivalency,  experiential  equivalency,  encompasses  how  two 
different  forms  of  a  test  or  survey  are  “experienced  by  the  examinee”  (Honaker,  1 988). 
These  experiences  can  be  “emotional,  perceptual,  and  attitudinal  reactions  to  the  two 
forms”  (Honaker,  1988).  In  other  words,  experiential  equivalency  means  that  a  paper- 
based  survey  and  computer-based  survey  can  be  considered  equivalent  only  if  the 
respondent  has  the  same  basic  experience  while  taking  the  surveys.  Because  this 
equivalency  is  qualitative  and  very  difficult  to  measure,  experience  differences  can  only 
be  assumed  to  a  large  degree. 

The  final  concept  of  equivalency,  relativity  of  equivalence,  examines  how 
familiar  the  survey  medium  is  to  the  respondent.  Relativity  of  equivalence  posits  that 
some  groups  may  have  less  familiarity  with  a  computer  than  another  group.  Because  of 
this  unfamiliarity,  this  group  will  respond  differently  than  the  group  that  is  familiar  with 
the  technology.  When  two  groups  respond  differently  to  the  same  survey  based  on  their 
familiarity  with  the  subject  matter,  format,  or  medium,  the  survey  cannot  be  considered 
equivalent  for  the  two  groups. 

Brennan  (1983)  remarks  that  standard  survey  and  test  instruments  that  are  being 
administered  in  both  paper  and  computer  formats  were  originally  validated  in  the  paper 
format  using  the  Classical  Test  Theory  methods.  Because  of  this,  it  is  important  to  study 
computer-based  instruments  using  identical  methods  (Wilson  et  ah,  1985:267).  He  also 
notes  that  by  “preserving  common  methods,  the  results  of  current  studies  can  be 
compared  to  prior  validation  studies”  meaning  that  paper  and  computer  surveys  need  to 
be  validated  using  the  same  methods. 
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Survey  Errors 

A  property  of  surveys  is  that  human  questions  measure  human  answers  (Fowler, 
1988).  Questions  are  poor  measures  because  they  are  often  unreliable,  inaccurate,  biased, 
or  cause  bias.  In  other  words,  they  introduce  error.  Not  only  do  questions  introduce  error, 
answers  often  introduce  as  much  or  more  bias  because  of  the  human  response  factor. 
Control  and  the  treatment  of  error  is  the  problem  crucial  to  all  types  of  surveys  (Hyman, 
1963).  Many  authors  write  about  many  types  of  errors:  emergent  error  (Hyman, 
1963:179);  sampling  error  (Fowler,  1988);  chance,  accidental,  constant,  and  non¬ 
compensating  errors  (Parten,  1950:403).  However,  three  main  groups  of  errors  can  be 
extracted  from  all  the  different  labels:  random,  biased  and  wrong  construct  errors.  These 
three  groups  of  errors  account  for  most  errors  that  can  be  found  while  conducting  a 
survey  and  will  be  fully  explained  in  the  following  paragraphs. 

Error  is  an  often  talked  about  concept  in  statistical  or  research  methods  textbooks. 
Dooley  (1995)  explains  that  any  observed  score  (X)  can  be  decomposed  to  X  =  T  +  E, 
where  T  is  the  true  score  component  and  E  is  the  error  component.  This  error  E  can  be 
any  factor  or  measure  that  makes  an  observed  score  different  from  the  true  score.  Dooley 
includes  the  random  error  (Er),  the  biased  error  ( B ),  and  the  wrong  construct  error  (Ew). 
Any  other  error  will  be  considered  too  insignificant  to  discuss  for  the  purposes  of  this 
study.  So,  the  new  formula  looks  like  this:  X  =  T  +  (Er+B+Ew). 

Random  errors  ( Er )  are  errors  that  randomly  occur,  such  as  errors  created  from 
guessing.  Because  they  are  random  with  both  positive  and  negative  effects,  they  will 
eventually  cancel  out  over  many  trials  (Dooley,  1995:79).  Random  errors  are  controlled 
through  two  methods:  First,  the  researcher  can  take  great  care  in  instrument  design  to 
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reduce  the  possibility  of  error.  Secondly,  the  researcher  can  increase  sample  size  to 
“obtain  the  desired  statistical  power”  (Hufnagel  &  Conca,  1994:50). 

Bias  error  ( B )  is  a  nonrandom  error  and  tends  to  be  more  complex  than  random 
error.  Bias  exists  when  an  “unmeasured  variable  skews  the  results  of  the  measurement 
process  in  a  systematic  way”  (Hufnagel  &  Conca,  1994:5 1).  Bias  can  be  introduced  by 
the  researcher,  the  measurement  instrument  itself,  such  as  a  questionnaire  or  test,  or  it  can 
be  introduced  based  on  a  particular  view  held  by  the  respondent.  A  bias  error  that  is  in 
the  instrument  is  less  significant  than  one  in  the  respondent  because  the  bias  will  be 
constant  across  all  respondents.  The  actual  observed  score  may  be  much  different  from 
the  true  score,  but  all  scores  will  be  consistently  affected.  The  respondent  bias  may  have 
to  do  with  likes,  dislikes  or  feelings  a  certain  person  or  group  has  toward  the 
measurement  instrument.  In  a  comparison  of  bias  from  a  paper-based  survey  to  a 
computer-based  survey,  bias  error  should  be  noticeable  as  a  difference  in  observed  scores 
between  the  two  methods.  A  constant  bias  will  assure  any  differences  are  consistently 
different  across  the  medium. 

The  third  error,  error  of  measuring  the  wrong  construct  ( Ew ),  is  a  major  concern  in 
the  discussion  of  equivalency  between  paper  and  computer  survey  instruments.  For 
example,  Webster  et  al.  (1996)  gave  a  paper  and  computer  survey  to  a  group  of  people 
who  had  recently  completed  a  computer  course.  The  survey  asked  the  subjects  about 
their  feelings  about  the  software  and  training.  Although  analysis  indicated  no  mean  or 
reliability  differences  between  the  two  methods  of  administration,  Webster  was 
concerned  that  significant  differences  found  in  construct  correlation  was  attributed  to 
those  subjects  who  took  the  survey  on  the  computer.  She  believes  they  paid  more 
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attention  to  the  computer  and  subsequently  the  constructs  such  as  computer  “playfulness” 
and  “experience”  took  on  different  meanings  compared  to  those  who  took  the  survey  by 
paper-and-pencil.  In  other  words,  the  subjects  may  have  answered  based  on  what  they 
were  currently  experiencing  on  the  computer.  Honaker  (1988:562)  states  that  if  the  two 
forms  “do  not  correlate  equally  well  with  relevant  external  criteria,  then  it  is  likely  that 
separate  constructs  are  being  measured.”  No  matter  how  well  the  researcher  tries  to 
control  for  bias,  some  bias  will  always  be  present.  Jaffe  and  Spirer  (1987:134)  state, 
“There  is  probably  no  way  to  eliminate  all  bias  from  all  questions  in  all  surveys.” 
Response  Bias 

Wiseman  (1972)  found  that  “responses  given  in  a  public  opinion  polling  are  not 
always  independent  of  the  method  used  to  collect  the  data.”  Response  bias  is  a  well- 
known  problem  in  survey  research,  which  researchers  must  constantly  consider  and  try  to 
control.  Paulhus  (1991)  defines  response  bias  as  a  “tendency  to  respond  to  a  range  of 
questionnaire  items  on  some  basis  other  than  the  specific  item  content.”  Other 
researchers  have  referred  to  response  bias  with  the  terms  “response  set”  (Cronbach, 
1946:475),  “response  effects”  (Kiesler  et  al.,  1986:404)  and  “response  style”  (Dooley, 
1995:90). 

Response  bias  may  be  due  to  a  person’s  personality,  culture,  education,  desire  for 
social  acceptance,  or  consistent  interference  from  the  medium  of  survey,  among  many 
other  reasons.  Response  bias  is  difficult  to  determine  from  surveys  or  tests  unless  there  is 
some  baseline  with  which  to  compare  it.  Response  bias  can  be  accounted  for  if  the 
means  or  variances  are  statistically  different  between  the  two  methods  (Honaker,  1988; 
Paulhus,  1991). 
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Cronbach  (1946)  defines  response  sets  as  “any  tendency  causing  a  person 
consistently  to  give  different  responses  to  test  items  than  he  would  when  the  same  content 
is  presented  in  a  different  form.”  Cronbach  (1946)  listed  six  response  sets  which 
influence  scores:  1)  tendency  to  gamble,  2)  definition  of  judgment  categories,  3) 
inclusiveness,  4)  bias;  acquiescence,  5)  speed  versus  accuracy,  and  6)  responses  on  essay 
tests.  These  six  response  sets  are  a  factor  of  personal  response  rather  than  group 
response.  Cronbach  (1946)  considers  them  a  bias  comparable  with  constant  errors.  Their 
affects  can  be  “reduced  by  any  procedure  that  increases  the  structuration  of  the  test 
situation”  (Cronbach,  1946:488). 

“Response  effects”  is  another  name  for  response  bias.  Kiesler  et  al.  (1986:404) 
defines  it  as  a  respondent’s: 

Systematically  refusing  to  answer  certain  questions,  or  giving  incomplete 
answers  or  not  following  instructions,  underreporting  socially  undesirable 
or  threatening  information,  over  reporting  socially  desirable  information, 
choosing  conventional  or  ‘moderate’  response  categories,  and  ‘yea- 
saying’ — agreeing  with  whatever  the  researcher  asserts. 

Kiesler  (1986:406)  asserts  that  computer  technology  will  reduce  these  effects  because  the 
setting  will  be  more  impersonal  and  anonymous,  and  that  respondents  “will  become  self- 
centered,  and  relatively  unconcerned  with  social  norms  and  with  the  impression  they  give 
others.”  In  contrast,  Reichard  (1998)  disagrees  by  arguing  that  a  “Big  Brother 
Syndrome”  affects  responses  because  of  the  focus  on  security  and  the  ability  of 
computers  to  track  and  possibly  identify  individual  users.  She  suggests  people  will  be 
more  concerned  with  social  norms  and  the  impression  they  give  others  because  the 
response  becomes  a  permanent  record  in  a  computer  system. 
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One  of  the  earliest  attempts  to  find  differences  in  responses  between  the  two 
mediums  was  conducted  by  Evan  &  Miller  (1969).  They  constructed  a  questionnaire 
based  on  the  popular  Minnesota  Multiphasic  Personality  Inventory  (MMPI)  and 
administered  it  by  both  computer  and  paper  to  60  MIT  undergraduate  students.  They 
found  that  subjects  answered  questions  more  truthfully  under  computer  administration 
when  the  questions  were  personal  or  disturbing  in  nature.  However,  they  found  no 
difference  if  the  questions  were  impersonal  or  emotionally  neutral,  suggesting  that  the 
computer  was  considered  an  impersonal  medium. 

A  1990  study  by  Lautenschlager  and  Flaherty  surveyed  241  psychology  students 
at  the  University  of  Georgia  using  psychological  honesty  scales  of  Impression 
Management  (IM)  and  Self-Deception  (SD).  They  concluded  that  IM  and  SD  scores 
were  directly  influenced  by  method  of  administration.  Computer  users  tended  to  produce 
lower  IM  and  SD  scores,  meaning  they  were  not  trying  to  make  themselves  look  better  to 
others  or  themselves.  Similarly,  George,  Lankford,  &  Wilson  (1992)  administered  both 
computer  and  paper  personality  surveys  to  97  undergraduate  student  volunteers.  Mean 
differences  were  found  between  the  paper  and  computer  administration  conditions. 
George  et  al.  summarized  that  “computer  anxiety  may  artificially  inflate  negative  affect 
scores  during  computer  administration”  (1992:203). 

Other  studies  looking  into  socially  desirable  responses  (the  extent  to  which  one 
answers  a  survey  to  present  the  most  desirable  image  to  another)  found  differences 
between  the  two  survey  methods,  but  with  unpredictable  results.  Kiesler  and  Sproull 
(1986)  presented  the  Marlowe-Crowne  Need  for  Approval  survey  to  100  students  and 
faculty  members  at  Camegie-Mellon  University.  The  found  that  answers  on  computers 
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were  less  socially  desirable  and  more  extreme  than  answers  on  paper.  In  contrast, 
Schuldberg  (1989)  reported  that  subjects  who  answered  surveys  by  computer 
administration  answered  with  more  social  desirability  answers  and  were  less  open  than 
subjects  using  paper  and  pencil.  The  paper  and  computer  surveys  produced  different 
results,  but  while  one  study  showed  more  socially  desirable  answers  on  paper,  the  other 
showed  more  socially  desirable  answers  on  the  computer  medium. 

Although  many  studies  show  a  link  between  survey  medium  and  response 
difference,  an  equal  number  indicate  no  differences  (Booth-Kewley  et  al.,  1992;  Paulhus, 
1991;  Lautenschlager  et  al.,  1990;  Honaker,  1988).  This  raises  the  need  to  determine 
what  possible  methodological  differences  may  have  caused  researchers  to  come  to  such 
opposite  conclusions.  Based  on  this  past  research  and  to  partially  answer  research 
question  one  (RQ1),  the  first  research  hypothesis  attempts  to  find  differences,  which  may 
indicate  bias,  in  overall  means  and  variances  between  paper  and  computer-based  surveys: 

HI  a)  Overall  means  and  variances  will  be  statistically  different  between 

computer-based  and  paper-based  surveys. 

Non-Response  Bias 

Although  not  the  opposite  of  response  bias,  non-response  bias  is  a  factor  that 
affects  response  rate.  In  short,  non-response  bias  is  bias  caused  by  a  difference  in  the 
attitudes  of  those  who  answer  a  survey  and  those  who  do  not  answer  a  survey.  Non¬ 
respondents  lower  the  response  rate  and  may  cause  an  abnormal  inflation  or  deflation  of 
the  responses  when  used  to  generalize  to  the  population.  Non-response  bias  is  caused  by 
people  not  willing  to  respond  to  a  survey  for  any  reason  in  a  voluntary  setting  when  their 
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views  may  be  consistently  different  from  the  response  group.  Singer  (1989:50)  warns, 
“non-response  can  seriously  bias  survey  estimates  and  distort  inferences.” 

Non-response  bias  may  have  to  do  with  a  person’s  preconceived  ideas  about  the 
survey,  questionnaire,  or  survey  medium  in  general,  so  the  person  may  not  even  attempt 
to  complete  the  survey.  Also,  the  person  may  attempt  or  start  the  survey,  but  not 
complete  it  for  some  reason.  Any  reason  for  not  completing  a  survey  is  considered  non¬ 
response  bias,  and  it  is  present  in  practically  any  self-report  survey  (Kiesler,  1986). 

What  if  the  majority  of  a  representative  sample  is  non-respondents,  and  the 
minority  are  respondents?  Fowler  (1988:48)  says,  “The  effect  of  non-response  on  survey 
estimates  depends  on  the  percentage  not  responding  and  the  extent  to  which  those  not 
responding  are  biased — that  is,  systematically  different  from  the  whole  population.”  The 
problem  with  non-response  is  not  actually  in  the  lack  of  responses,  but  is  created  if  those 
responding  are  different  from  the  population.  Non-response  can  exacerbate  the 
difference  between  the  sample  and  population.  For  example,  a  survey  may  have  been 
distributed  to  a  random,  representative  sample  of  the  population,  but  if  the  non-response 
is  high  enough,  it  is  unknown  if  the  true  views  of  the  population  can  be  assessed  by  the 
small  potentially  biased  sub-sample. 

There  are  two  forms  of  non-response  bias  with  equally  damaging  possible  impacts 
on  a  study.  The  first  is  unwillingness  to  answer  and  has  to  do  with  a  person  not 
responding  because  the  person  does  not  want  to.  This  person  has  the  choice  to  complete 
the  survey  or  not,  and  decides  not  to  complete  it  for  any  personal  reason.  The  second 
form  of  non-response  bias  is  inability  to  answer  and  has  to  do  with  someone  not  being 
able  to  fill  out  a  survey  for  some  reason.  This  may  entail  the  lack  of  availability  or 
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accessibility  of  the  survey  medium.  In  1936,  the  Literary  Digest  Poll  failed  because  the 
organizers  only  polled  people  with  phones.  What  about  people  without  phones?  The 
same  can  be  said  about  computers.  What  about  potential  respondents  without  computers? 
This  is  probably  the  most  talked  about  problem  in  computer  survey  administration  today. 
Some  researchers’  sampling  frames  only  contain  those  individuals  who  have  access  to  a 
computer  (Rosenfeld  et  al.,  1993).  Some  polls,  found  on  websites  such  as  CNN.com,  can 
only  be  accessed  by  those  using  a  computer  with  an  Internet  connection.  How 
generalizable  are  the  results  of  a  poll  whose  only  participants  are  computer  users? 

A  theory  explaining  response  rates  in  Air  Force  members  is  found  in  Adams’ 
(1996:25)  unpublished  study.  Her  research  found  that  Air  Force  personnel  have  higher 
response  rates  for  paper  surveys  compared  with  computer  (e-mail)  surveys.  She 
theorized  that  they  “perceive  written  communication  as  more  formal  and  task-oriented.” 
Because  it  is  perceived  as  more  formal,  the  response  rate  for  paper  surveys  is  greater  than 
for  computer  surveys.  Conversely,  the  non-response  rate  for  computer  surveys  was 
higher  than  for  paper  surveys. 

Sjostrom  (1999)  conducted  research  to  measure  non-response  error  and  incorrect 
answer  errors  in  a  survey  sent  to  4000  Swedish  citizens.  His  calculated  response  rate  was 
43%  compared  with  a  62%  response  rate  for  another  survey  conducted  similarly  to  the 
same  population.  Sjostrom  was  able  to  compare  the  objective  responses  to  archived 
historical  data  to  determine  whether  errors  existed  in  the  responses.  Because  of  the 
relatively  low  response  rate,  he  concluded  that  non-response  bias  accounted  for  66%  of 
the  error  in  his  survey  results  when  generalized  to  the  population. 
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To  fully  answer  research  question  one  (RQ1)  and  to  determine  if  non-response 

bias  is  a  significant  factor  between  paper  and  computer  surveys,  the  second  research 

hypothesis  attempts  to  determine  equivalency  based  on  response  rates: 

Hlb)  Overall  computer-based  survey  response  rates  will  be  statistically 
different  than  paper-based  response  rates. 

Format  Differences 

Some  research  has  argued  that  format  differences  can  affect  user  responses. 
Format  differences,  in  this  case,  refer  to  surveys  with  identical  questions  but  different  in 
question  order,  response  type,  font  (design),  or  overall  complexity  rather  than  differences 
between  the  methods  of  administration.  Webster  et  al.  (1996)  reasoned  that  differences 
discovered  between  paper  and  computer  surveys,  in  some  studies,  could  possibly  be 
attributed  to  the  format,  design,  or  complexity  differences  between  two  surveys.  In  other 
words,  researchers  may  not  have  controlled  for  slight  format  differences  making  it  appear 
that  the  any  difference  was  caused  by  the  method  of  administration  rather  than  format 
differences.  Cizek  (1994)  found  that  even  the  slightest  change  in  answer  order  in  a 
multiple-choice  test  significantly  affected  responses  between  otherwise  identical  tests. 
Moving  the  correct  answer  only  two  positions  created  unpredictable  but  “statistically 
significant  differences”  (Cizek,  1994:18).  He  cautioned  test  creators  to  avoid  reordering 
options  on  similar  examinations  or  risk  creating  unequivocal  tests.  Beaton  and  Zwick 
(1990)  suggested  that  differences  between  response  types  (i.e.  circling  a  correct  answer 
versus  filling  in  bubbles  for  correct  answers)  affect  how  respondents  answer  survey  or 
test  items. 
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Webster  et  al.  (1996:568)  states,  “Research  attention  that  compares  modes  of 
administration  while  minimizing  format  differences  is  needed.”  Such  format  and 
complexity  differences  are  easily  implemented  on  computers  (inability  to  backtrack, 
increased  attention  on  items  when  individually  presented,  immediate  and  automatic 
feedback,  initial  cursor  positioning,  and  survey-taker  feelings  of  no  control  when 
previous  answers  cannot  be  changed).  It  does  not  make  sense  to  compare  responses 
between  a  paper  questionnaire  of  one  design  with  a  computer  questionnaire  of  another 
design.  If  the  designs  are  different,  how  can  the  researcher  tell  if  any  differences  are  due 
to  the  medium  or  due  to  the  format  or  complexity  differences?  When  comparing 
differences  in  medium  of  survey  or  test  administration,  it  is  vital  to  keep  format 
differences  minimized  (Webster  et  al.,  1996;  Comley,  2000;  and  others).  Although 
researchers  have  recognized  this  fact,  in  many  studies  (Booth-Kewley  et  al.,  1992; 
George  et  al.,  1992;  Kawasaki  et  al.,  1995;  and  others),  the  researchers  did  not  reveal 
whether  format  or  complexity  differences  were  controlled  or  not.  If  differences  were 
controlled,  this  was  important  methodological  information  to  exclude. 

The  following  research  hypotheses  attempt  to  answer  research  question  two 
(RQ2)  to  determine  if  complexity  or  format  differences  introduce  significant  response 
bias  into  survey  results: 

H2a)  Overall  means  and  variances  will  be  statistically  different  between  two 
computer-based  surveys  of  different  complexity. 

H2b)  Overall  survey  response  rates  will  be  statistically  different  between  two 
computer-based  surveys  of  different  complexity. 
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Gender  and  Computers 

Why  might  certain  groups  display  more  response  bias  toward  the  computer  than 
another  group?  Gefen  and  Straub  (1997)  indicate  that  a  person’s  gender  affects  how 
computers  and  computer  technology  is  perceived.  Although  they  found  no  difference  in 
men’s  and  women’s  use  of  electronic  mail,  the  study  raised  questions  about  perception 
differences  between  the  sexes  that  may  occur  when  faced  with  a  computer-based  survey 
as  opposed  to  a  paper-based  survey. 

Venkatesh  and  Morris  (2000: 1 15)  asserted  that  gender  differences  play  a  role  the 
“individual  adoption  and  sustained  usage  of  technology.”  Their  study,  which  focused  on 
gender  differences  in  the  linkages  of  the  determinants  of  technology  use  and  acceptance 
from  Davis’  (1989)  Technology  Acceptance  Model  rather  than  mean  gender  differences 
in  use  and  acceptance,  found  that  women  weighed  the  determinant  “perceived  ease  of 
use”  (Davis,  1989)  as  a  direct  factor  of  “behavioral  intention”  (Davis,  1989)  more 
strongly  than  men.  In  other  words,  women’s  intentions  to  use  technology  is  influenced 
by  their  perception  of  how  easy  it  is  to  use  that  technology  to  a  greater  degree  than  men. 
An  important  point  that  can  be  reasoned  here  is  that  when  a  man  and  a  woman  are  both 
provided  an  opportunity  to  take  part  in  a  computer-based  survey,  the  woman’s 
willingness  to  participate  in  that  survey  will  be  more  heavily  determined  by  her 
perceptions  of  the  ease  of  use  of  that  computer  survey  than  the  man. 

Which  gender  might  perceive  a  computer  as  easier  to  use?  Edwards  (1990:107) 
suggests  that  men  perceive  computers  as  more  friendly  and  familiar  than  women  because 
men  think  in  the  same  modes  as  computers:  “syllogistic,  quasi-mathematical  logic  and 
formal  gaming.”  Edwards  (1990:125)  also  postulates  that  “computers  do  not  simply 
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embody  masculinity;  they  are  culturally  constructed  as  masculine  mental  objects.” 
Because  men  perceive  computers  as  more  friendly  and  familiar  than  women  do,  women 
may  be  more  likely  to  decide  not  to  participate  in  a  computer-based  survey  at  a  higher 
rate  compared  with  men. 

Canada  &  Brusca  (1991),  Kramer  &  Lehman  (1990),  and  Whitley  (1996)  agree 
that  men  have  been  more  socially  conditioned  to  perform  well  on  computers  compared 
with  women.  Whitley  (1996)  says  “computer  use  in  schools  has  been  linked  to 
traditionally  ‘masculine’  subjects  as  science  and  mathematics  but  not  to  traditionally 
‘feminine’  subjects  such  as  art  and  literature.”  However,  his  study  showed  only  a  small 
difference  between  men’s  and  women’s  anxiety  levels  and  computer-related  behavior 
(Whitley,  1996).  Kramer  et  al.  (1990)  showed  that  “gender-related  differences  in 
learning  and  using  computers  are  documented  at  all  educational  levels.”  Canada  et  al. 
(1991)  concludes  from  her  study  that  a  “technical  gender  gap”  does  exist  and  may  cause 
women  to  be  less  likely  to  meet  the  technological  challenges. 

Because  women  tend  to  relate  to  and  perceive  computer  technology  different  than 
men,  it  is  believed  that  women  will  respond  differently  to  computer  and  paper-based 
surveys.  It  is  further  believed  that  because  men  are  more  socially  conditioned  to  use  and 
perform  well  with  computer  technology,  no  differences  will  be  found  between  responses 
and  response  rates  between  the  two  surveys  for  men.  The  following  four  research 
hypotheses  attempt  to  answer  part  of  research  question  three  (RQ3)  which  asks  whether 
gender  affects  responses  between  paper  and  computer-based  surveys: 
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H3a)  Women  will  have  statistically  different  response  means  and  variances 
between  the  simple  computer,  complex  computer  and  paper-based  surveys. 

H3b)  Men  will  have  statistically  similar  response  means  and  variances 
between  the  computer-simple,  computer-complex  and  paper-based  surveys. 

H3c)  Women  will  have  a  statistically  different  response  rate  between  the 
paper  and  computer-based  surveys. 

H3d)  Men  will  have  a  statistically  similar  response  rate  between  the  paper 
and  computer-based  surveys. 

Military  Commission,  Education  and  Computers 

This  study  looks  at  differences  in  individuals  of  different  military  commission 
(officer  versus  enlisted)  because  this  is  an  important  and  obvious  distinction  among 
military  personnel.  How  differently  might  officers  and  enlisted  personnel  respond  to 
computer  and  paper-based  surveys?  Because  no  prior  research  was  found  that  linked 
military  commission  to  computer  use  and  acceptance,  a  link  can  only  be  drawn  from 
gathered  inferential  evidence. 

The  differences  between  individuals  in  the  officer  force  and  enlisted  force  are 
numerous.  All  officers  have  a  bachelor’s  degree  or  higher.  According  to  the  Air  Force 
FY2000,  first  quarter  demographic  report  (AFPC,  2000),  only  4.7  percent  of  enlisted 
members  have  a  bachelor’s  degree  or  higher.  The  officer  force’s  average  age  is  35  and 
the  enlisted  force’s  average  age  is  29  years  old.  Most  officers  receive  rigorous  leadership 
and  command  training  from  basic  officer  training  through  intermediary  and  advanced 
leadership  and  command  training.  Enlisted  members  receive  some  leadership  training  at 
various  stages,  but  not  command  training.  Lastly,  most  officers  are  placed  in  leadership 
or  command  jobs  from  the  beginning  of  their  commission.  Enlisted  members  normally 
start  their  enlistment  as  trainees  with  few  responsibilities  and  slowly  receive  more 
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responsibility  as  time  passes.  This  difference  in  responsibility  between  the  two  forces 
places  each  group  into  different  social  classes  that  can  be  likened  to  the  working  classes 
of  white-collared  workers  and  blue-collar  workers. 

Which  factors  might  influence  how  these  two  groups  respond  to  computer  and 
paper-based  surveys?  Education  appears  to  be  the  most  distinct  difference  between  the 
groups,  and  there  is  some  empirical  evidence  that  suggests  that  technology  and  computer 
use  is  affected  by  education  level.  Even  reason  seems  to  suggest  that  education  level 
affects  a  person’s  perceptions  about  technology  and  computer  use.  If  computers  are  used 
in  education,  the  more  educated  a  person,  the  less  anxiety  and  more  familiarity  he  or  she 
should  have  compared  to  someone  who  did  not  spend  that  time  around  computers.  Even 
without  much  computer  use  in  education,  education  and  increased  knowledge  alone  seem 
to  enable  a  person  to  be  more  accepting  of  innovations  and  technology  (Rogers,  1995). 
Rogers  (1995)  says  that  formal  educational  background  has  a  significant  effect  on  IT  use 
and  acceptance.  Rogers  states  that  the  more  years  of  formal  education  a  person  has,  the 
sooner  he  or  she  will  adopt  IT  innovations. 

The  other  major  factor  that  may  influence  differences  in  officer  and  enlisted 
responses  is  social  structure  or  class.  Because  the  officer  and  enlisted  forces  have 
traditionally  been  separated  into  their  own  separate  social  class  based  on  income, 
occupation,  education,  and  responsibility  (Cotton,  1994),  it  is  important  to  consider  these 
differences  beyond  simple  educational  differences.  Cotton  (1994:409)  says  that  social 
class  should  a  part  of  the  “normal  inventory  of  independent  and  mediating  variables  used 
by  organizational  behavior  researchers  and  theorists.  Possible  relationships  have  been 
identified  that  can  occupy  the  attention  of  a  number  of  researchers.”  The  present  point  is 
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to  show  that  social  class  (officer  versus  enlisted)  is  an  important  consideration  in 
studying  differences  within  the  Air  Force.  Rogers  (1995:269)  validates  this  importance 
by  stating  that  those  with  “higher  social  status”  and  a  “greater  degree  of  upward 
mobility”  will  adopt  new  technologies,  like  web-based  interfaces,  much  more  quickly 
than  those  without  a  high  social  status  of  large  degree  of  upward  mobility.  Other 
characteristics  of  people  that  accept  and  adopt  IT  more  quickly  and  easily  are  those  with: 

•  greater  rationality 

•  greater  intelligence 

•  more  favorable  attitude  toward  change 

•  greater  ability  to  cope  with  uncertainty 

•  higher  aspirations 

•  greater  knowledge  of  innovations 

•  higher  degree  of  leadership  (Rogers,  1995:273) 

These  characteristics  are  not  the  separating  factors  between  members  of  the 
officer  and  enlisted  force.  However,  they  do  give  an  idea  of  the  type  of  attributes  a 
person  might  possess  who  is  more  accepting  of  IT.  These  characteristics  tend  to  be  those 
of  leaders,  and  officers  are  the  leaders  within  the  Air  Force.  Because  of  these  factors 
listed  and  the  general  differences  that  can  be  roughly  assumed  between  officers  and 
enlisted  personnel,  it  is  believed  that  enlisted  personnel  will  respond  differently  to 
computer  and  paper-based  surveys.  It  is  further  believed  that  because  officers  do  have 
educations  that  are  more  formal  and  are  in  higher  leadership  positions  than  the  enlisted 
force,  no  differences  will  be  found  between  responses  and  response  rates  between  the  two 
surveys  for  officers.  The  following  four  research  hypotheses  attempt  to  answer  the  last 
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part  of  research  question  three  (RQ3)  which  asks  whether  military  commission  affects 

responses  between  paper  and  computer-based  surveys: 

H4a)  Enlisted  personnel  will  have  statistically  different  response  means  and 
variances  between  the  simple  computer,  complex  computer  and  paper-based 
surveys. 

H4b)  Officers  will  have  statistically  similar  response  means  and  variances 
between  the  simple  computer,  complex  computer  and  paper-based  surveys. 

H4c)  Enlisted  personnel  will  have  a  statistically  different  response  rate 
between  the  paper  and  computer-based  surveys. 

H4d)  Officers  will  have  a  statistically  similar  response  rate  between  the 
paper  and  computer-based  surveys. 

Summary 

Based  on  this  review  of  the  relevant  literature,  several  conclusions  were  drawn 
which  helped  shape  the  design  and  objectives  of  the  current  study.  Most  importantly, 
whether  the  survey  is  psychological  or  organizational  in  nature,  research  findings  have 
been  mixed.  Some  findings  of  nonequivalence  between  the  two  methods  of 
administration  found  the  computer  to  increase  some  response  bias,  while  other  studies 
found  the  computer  to  decrease  some  response  bias.  This  further  muddles  the  question  of 
what  effect  the  computer  actually  plays  in  the  introduction  of  response  bias. 

Secondly,  most  of  the  research  done  has  been  conducted  in  a  controlled 
environment.  Subjects  were  given  some  type  of  stimuli  (i.e.  computer  training  program) 
and  then  asked  to  fill  out  a  questionnaire  about  the  stimuli.  Other  surveys,  mostly  of  a 
psychological  nature,  were  conducted  under  experiment  like  conditions.  Only  a  couple 
researchers,  in  the  review,  allowed  the  subjects  to  fill  out  paper  and  computer  surveys 
anonymously  and  without  pressure. 
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A  third  conclusion  concerns  the  lack  of  non-response  factors  found  in  the 
previous  research.  This  is  important  because  non-response  is  a  potentially  bias 
introducing  factor  in  most  organizational  surveys  today.  Without  specifically  considering 
the  non-response  bias  in  a  study  on  survey  method  equivalence,  it  is  impossible  to  tell 
whether  a  paper  format  and  computer  format  are  indeed  equivalent  on  nonequivalent. 

Fourth,  format  and  complexity  differences  (e.g.  font,  colors,  spacing,  questions 
per  page,  error  checking  mechanisms)  were  largely  ignored  in  most  studies.  Only 
Webster  et  al.  (1996)  focused  on  format  differences  as  a  possible  factor  in  non¬ 
equivalency  findings. 

Lastly,  no  studies  focused  on  either  gender  or  military  commission/education 
level  differences.  Any  differences  that  occur  between  these  groups  may  potentially  skew 
findings,  if  the  differences  are  not  recognized.  For  example,  Booth-Kewely  et  al.  (1992) 
only  tested  men  Navy  recruits  finding  no  differences,  but  other  researchers  who  studied  a 
mix  of  men  and  women  found  differences.  Without  specifically  considering  group  biases 
it  is  impossible  to  make  accurate  findings. 
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III.  Methodology 


Chapter  Overview 

This  chapter  explains  the  methodology  that  was  used  to  answer  the  research 
questions  and  test  the  research  hypotheses  presented  in  the  previous  two  chapters.  Based 
on  the  literature  review  of  chapter  two,  this  study  made  every  effort  to  design  and 
conduct  an  experiment  that  used  the  best  thoughts  from  previous  research,  while  avoiding 
several  pitfalls  that  befell  the  previous  studies.  This  chapter  explains  the  theory, 
experimental  design,  research  methods,  survey  development,  population  and  sample, 
survey  administration,  and  analytical  methods  used  to  test  the  hypotheses. 

Experimental  Design 

This  research  design  attempts  to  learn  about  Air  Force  members’  responses  and 
response  rates  of  surveys  using  a  survey.  The  survey,  in  this  case,  becomes  the 
experimental  manipulation,  while  the  observation  becomes  the  analysis  of  responses  and 
response  rates.  Using  a  voluntary  survey  appears  to  be  the  most  appropriate  way  to 
assess  possible  bias  based  on  the  media  of  the  survey,  rather  than  through  some  other 
experimental  or  observational  technique.  A  key  assumption  is  that  people  will  complete 
a  survey  they  “like”  and  not  complete  a  survey  they  “don’t  like.” 

Based  on  Dooley  (1995),  this  experimental  design  is  composed  of  a  three-factor 
true  experiment  using  cross-sectional  surveys  (Figure  1).  The  experimental  design 
involved  sending  one  of  three  differing  surveys  to  900  random  active-duty  Air  Force 
personnel  composed  of  a  50/50  men/women  mix  and  a  50/50  officer/ enlisted  mix.  Each 
survey  was  composed  of  identical  questions  and  was  provided  to  300  respondents 
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through  a  similar  method.  Three  surveys  for  300  people  were  decided  upon  based  on 
time  and  resource  limitations  and  the  expectation  of  a  25-35%  response  rate.  This  was 
hoped  to  produce  a  high  enough  sample  size  (N  =  90  for  each  survey  and  N  =  45  for  each 
demographic  stratification),  which  was  sufficient  to  meet  the  criteria  for  a  statistically 
large  sample  (McClave,  1998). 


XCS  O 


R  Xcc  0 

XP  O 


XCs  •  Computer-simple  -  Computer-based  and  similar  in 
appearance  to  paper-based  survey 

XCc  ■  Computer-complex  -  Computer-based  with  complex 
programmatic  controls 

XP  :  Paper-based  survey 

1 _ _ _ 1 

Figure  1.  Experimental  Design. 


Referring  to  Figure  1,  one  of  the  three  groups  of  questionnaires  was  given  a 
paper-based  (XP)  survey.  As  the  most  traditional  form  of  questionnaire,  it  was 
administered  to  the  control  group.  The  second  survey  (Xcs)  was  computer-based, 
accessed  over  the  Internet  via  a  web  browser,  simple  (cs  =  computer-simple)  and  similar 
in  format  to  the  paper-based  survey.  The  third  survey  (Xcc)  was  also  computer-based  but 
was  complex  in  format  (cc  =  computer-complex)  and  had  programmatic  controls  as 
opposed  to  the  other  computer  survey  to  maximize  visual  and  processing  differences 
while  keeping  the  questions  the  same. 

The  experimental  objective  was  to  make  the  first  two  surveys  as  identical  as 
possible,  only  differing  in  medium  of  administration.  The  paper-based  survey  was  an 
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exact  replica  of  the  computer-based  version  in  regard  to  style,  font,  color,  and  spacing. 
The  differences  were  minimized,  to  the  maximum  extent  possible,  so  that  any  differences 
observed  between  the  two  methods  were  due  strictly  to  the  method  of  administration.  For 
the  third  computer-based  survey,  it  was  important  to  maximize  the  differences  between 
the  other  two  surveys.  It  was  colorful,  as  opposed  to  the  paper  and  first  survey,  which 
were  plain  black  and  white,  and  displayed  different  fonts  and  a  different  layout.  Also, 
programmatic  controls  were  installed  to  ensure  user  input  was  valid  and  formatted 
correctly.  Lastly,  the  overall  complexity  was  increased  by  directing  the  respondent 
through  several  pages  of  survey  material  and  only  allowing  the  respondent  to  proceed 
when  all  data  was  correctly  and  fully  input.  The  goal  was  to  discover  any  differences 
introduced  by  a  complex  computer-based  format.  As  explained  in  the  literature  review, 
prior  research  cited  format  differences  as  a  reason  for  dissimilarity  encountered  in  user 
responses  (Webster  &  Compeau,  1996).  The  current  study  examined  both  simple  and 
complex  formats  simultaneously  to  determine  if  any  significant  differences  in  responses, 
which  may  indicate  bias,  existed  between  differing  formats. 

Research  Method 

The  research  method  consisted  of  the  following  steps  based  on  the  chosen 
experimental  design.  The  remainder  of  this  chapter  will  explore  each  step  in  depth. 

1)  develop  the  survey  instruments, 

2)  obtain  permission  to  conduct  the  survey  from  Air  Force  Survey  Branch, 

3)  select  subjects  from  the  population, 

4)  conduct  pilot  study, 

5)  administer  surveys, 

6)  gather  survey  results,  and 

7)  perform  statistical  analysis  of  the  final  data. 
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Survey  Development 

Because  this  study  was  primarily  concerned  with  user  perceptions  of  the  survey 
instrument  based  on  the  medium  of  administration,  the  actual  content  of  the  survey 
instrument  was  less  significant  than  in  most  survey  research.  Each  survey,  Xp  (paper- 
based),  Xcs  (computer-similar),  and  Xcc  (computer-complex)  provided  exactly  the  same 
instructions,  demographic  sections,  and  questions.  Providing  exact  content  attempted  to 
rule  out  response  differences  being  contributed  to  differences  in  questions. 

The  demographic  section  was  composed  of  two  main  areas,  mandatory  and 
optional  data  entry  fields.  The  mandatory  fields  were  gender,  rank,  AFSC,  Major 
Command,  and  education  level.  The  optional  fields  were  marital  status,  number  of 
dependents,  years  at  residence,  home  of  record  and  yearly  income.  The  mandatory  area 
was  important  for  delineation  among  respondents.  Although  the  survey  required  input  of 
specific  rank,  the  analysis  section  will  only  delineate  between  enlisted  and  officer.  The 
survey  collected  as  many  data  as  possible,  but  because  of  an  expected  small  N,  it  was 
necessary  to  stratify  the  data  into  bi-variate  groups  per  variable  under  study.  The 
optional  section  was  designed  to  determine  if  sensitivity  of  question  affected  self¬ 
disclosure  between  methods  of  administration. 

The  second  section  consisted  of  10  Likert-style  statements.  These  were 
statements  about  the  respondent’s  views  of  the  Air  Force,  in  terms  of  organizational 
commitment,  and  required  each  respondent  to  agree  or  disagree  with  the  statements.  The 
10  statements  are  a  subset  of  Mowday,  Steers,  &  Porter’s  (1979)  Organizational 
Commitment  Questionnaire.  The  organizational  commitment  scale  had  demonstrated 
satisfactory  test-retest  reliability  (r)  =  .53  -  .75,  internal  consistency  (coefficient  a)  =  .82 
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-  .93,  and  convergent  validity  =  .63  -  .74  (Mowday  et  al.,  1979).  The  study  was  designed 
to  avoid  any  reference  to  computers  or  computer  technology.  Webster  and  Compeau 
(1996)  concluded  that  computer-based  surveys  that  ask  questions  about  computers  might 
introduce  bias  because  the  respondent  pays  more  attention  to  the  computer. 

The  computer-based  survey  consisted  of  a  web  page  with  an  address  of 
http://en.afit.af.mil/research2000/surveycs.asp  or  surveycc.asp  depending  on  whether  it 
was  the  simple  or  complex  format.  The  programming  language  used  to  create  the  survey 
was  Active  Server  Pages  3.0.  User  responses  consisted  of  typed  items  and  “point-and- 
click”  items.  Once  the  user  completed  the  survey,  he  or  she  pressed  the  “submit”  button. 
When  the  “submit”  button  was  pressed,  all  the  data  on  the  page  was  saved  to  a  Microsoft 
Access  97  database  residing  on  the  web  server.  The  user  then  saw  a  screen  explaining 
that  the  data  was  saved.  The  programming  code  ensured  that  no  computer  could  submit  a 
survey  more  than  once.  In  addition,  the  code  saved  the  date,  time,  and  Internet  Protocol 
address  to  the  corresponding  record.  However,  the  program  did  not  perform  error 
checking  of  the  data  on  the  computer-based  similar  format,  so  blank  items  were  allowed. 
This  ensured  consistency  with  the  paper-based  questionnaire,  which  did  not  have  error 
checking.  The  computer-based  complex  did  have  error  and  validity  checking  with  “pop¬ 
up”  indication  messages  to  the  respondent.  This  technology  was  only  available  because 
of  the  computer  medium  and  had  no  paper-based  equivalent  beside  human  intervention. 
Permission  to  Conduct  Survey 

The  Air  Force  Survey  Branch  at  the  Air  Force  Personnel  Center  (HQ 
AFPC/DPSAS)  approves  all  surveys  that  are  administered  to  active-duty  Air  Force 
personnel  without  specific  commander  consent.  Once  the  survey  was  developed,  it  was 
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provided  along  with  justification  for  the  survey  to  the  Air  Force  Survey  Branch.  It  was 
approved  on  8  May  2000  with  an  AFPC  Survey  Control  Number  of  SCN  00-34  with  an 
expiration  on  3 1  December  2000.  The  SCN  granted  authority  to  randomly  select  and 
survey  Air  Force  personnel  based  on  the  prearranged  agreement  with  the  AFPC  Survey 
Branch.  Air  Force  Instruction  36-2601  governs  Air  Force  survey  procedures  and  was 
followed.  Additionally,  permission  was  obtained  to  use  the  organizational  commitment 
scales  from  the  author  Richard  T.  Mowday  on  1 7  April  2000. 

Population  and  Sample 

The  population  for  this  study  was  all  Air  Force  active-duty  personnel  from 
Airman  Basic  to  Colonel.  General  Officers  were  excluded  as  a  courtesy  to  their  position 
and  responsibilities.  To  obtain  a  random  sample  of  900  Air  Force  active-duty  personnel 
(450  men  and  450  women,  consisting  of  450  officers  and  450  enlisted),  an  ATLAS  listing 
was  requested  from  the  demographics  section  at  AFPC.  AFPC  provided  a  random 
sample  of  a  50/50  mix  based  on  gender  and  rank  on  1 1  May  2000.  The  listing  included 
name,  rank,  MAJCOM  and  duty  address. 

Pilot  Test 

A  pilot  test  was  conducted  to  ensure  the  understandability,  usability,  and  internal 
reliability  of  the  paper-based  and  computer-based  surveys.  This  involved  administering 
the  three  questionnaire  types  (paper,  computer-simple,  and  computer-complex)  to  AFIT 
graduate  students  in  the  Information  Resource  Management  program.  Results  from  the 
pilot  test  helped  refine  the  final  survey  instruments,  cover  sheets,  and  administration 
procedures.  Most  significantly,  two  questions  were  reworded  based  on  some  comments 
of  confusion  from  the  pilot  group,  and  the  cover  sheets  were  reworded  to  produce  a  more 
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pleasing  tone.  Additionally,  some  errors  occurred  on  the  web  pages  that  required 
reprogramming  and  other  slight  modifications. 

Although  a  Cronbach’s  alpha  measurement  of  .54  was  calculated  as  the  internal 
reliability  of  the  questions,  no  questions  were  subsequently  removed.  Removing  any  one 
question  would  have  lowered  the  alpha  level.  It  was  believed  that  the  low  alpha  was  a 
result  of  a  relatively  low  number  of  participants  in  the  pilot  test  (N=  18)  and  the  high 
probability  of  bias  as  a  result. 

Survey  Administration 

Nine  hundred  number  10  white  envelopes  were  addressed  and  validated  by  the 
United  States  Postal  Service  based  on  the  listing  from  AFPC.  Three  hundred  paper 
questionnaires  were  created  based  on  a  printout  of  the  survey  web  page.  A  cover  page 
accompanied  each  paper-based  questionnaire  explaining  the  reason  for  the  survey,  asking 
for  the  respondent’s  cooperation,  and  providing  instructions  for  returning  the  survey  in 
the  pre-addressed  return  envelopes  within  two-weeks.  The  return  postage  was  free  to 
respondents  because  the  envelopes  traveled  through  official  mail  channels.  The  cover 
letters  and  surveys  were  randomly  assigned  to  three  hundred  number  10  envelopes  along 
with  the  pre-addressed  number  9  return  envelopes.  The  other  600  personnel  received 
one  of  two  other  instruction  sheets.  Both  instruction  sheets  matched  the  paper-based 
survey’s  cover  sheet,  but  directed  the  user  to  go  to  a  computer  and  type  the  Internet 
address  into  the  computer’s  browser  within  2  weeks  of  receipt.  The  Xcs  computer-based 
survey  was  named  “surveycs.asp”  while  the  XCc  computer-based  survey  was  named 
“surveycc.asp.”  In  addition,  a  final  instruction  on  the  cover  letter  asked  the  respondent  to 
return  the  included  pre-addressed  postage-free  4”  by  6”  postcard,  if  he  or  she  did  not  have 
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access  to  a  computer.  The  postcard  had  space  for  the  respondent  to  list  the  required 
demographic  data.  If  the  respondent  had  a  computer,  the  instructions  asked  him  or  her  to 
simply  discard  the  postcard. 

Gather  Survey  Results 

Approximately  two  months  were  allowed  from  the  first  mailing  day  before 
analyzing  data.  The  computer-based  survey  database  was  checked  daily  to  make  sure 
there  were  no  problems  with  access  or  corruption.  As  computer-based  surveys  were 
submitted,  an  automatic  e-mail  message  was  sent  to  the  researcher’s  AFIT  e-mail  account 
so  response  rate  could  be  continually  monitored.  Completed  paper  surveys  were 
collected  at  the  AFIT/ENV  organizational  box  as  needed. 

All  paper  survey  data  were  manually  updated  into  Microsoft  Excel  2000.  Each 
response  sheet  was  individually  coded  into  a  spreadsheet  row.  The  computer-based 
responses,  which  were  collected  through  a  web-based  Microsoft  Access  97  database, 
were  imported  into  the  same  Microsoft  Excel  2000  spreadsheet  for  analysis. 

Analytical  Methods  (Statistics) 

The  first  analytical  test  of  the  data  involved  comparing  response  rates.  For  each 
method  (paper,  computer-simple,  and  computer-complex)  the  response  rate  was  obtained 
by  dividing  the  number  received  by  the  number  sent.  The  number  sent  for  each  method 
was  not  300,  since  approximately  15-20  of  each  survey  was  returned  because  of 
personnel  moves  or  insufficient  addresses.  Next,  the  assumptions  of  normality  and  large 
sample  were  verified.  Independence  was  assumed  for  the  proportions  since  the  samples 
for  each  survey  are  unrelated  to  the  other  samples.  Next,  the  three  population  proportions 
were  compared  based  on  meeting  these  three  assumptions.  This  first  test  examined 
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whether  the  proportion  of  respondents  was  statistically  equivalent.  For  this  test,  and  the 
following  tests,  a  two-tailed  hypothesis  test  with  the  Type  I  error  rate  set  at  alpha  (a)  = 
.10  was  used  to  test  for  equivalency.  Since  this  research  was  exploratory  in  nature,  this 
relatively  high  alpha  level  was  deemed  appropriate  based  on  past  exploratory  research 
alpha  levels  (Arbaugh,  2000;  James  &  Wortring,  1995).  First,  the  paper-based  survey  was 
compared  to  both  computer-based  surveys.  Then  the  two  computer-based  surveys  were 
compared  together.  Microsoft  Excel  2000  aided  in  calculations  of  the  formula  shown  in 
Figure  2. 
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Test  Statistic:  z  = 
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Figure  2.  Two-Tailed  Test  for  statistical  difference  (McCIave,  1998). 

The  results  of  this  test  told  whether  or  not  the  response  rates  were  equivalent.  Next,  any 
differences  among  gender  and  military  commission  were  tested.  These  tests  were  the 
same  as  shown  in  Figure  2.  The  test  determined  if  a  statistical  difference  between  two 
proportional  groups  existed  once  the  assumptions  of  normality,  large  samples,  and 
independence  were  verified.  The  goal  was  to  find  a  statistically  significant  difference  in 
response  rates  across  these  groups. 

After  completion  of  the  proportion  tests,  responses  to  the  Likert-style  questions 
were  assessed  between  the  three  surveys  and  between  gender  and  military  commission. 
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The  ten  organizational  commitment  scales  were  added  together  to  provide  a  composite 
score  for  each  person.  This  composite  score  per  person  was  calculated  for  each 
questionnaire  using  Microsoft  Excel  2000.  The  statistical  package  JMP  Version  3.2.6 
was  used  to  calculate  the  Analysis  of  Variance  (ANOVA)  to  determine  if  a  statistical 
difference  in  the  survey  answers  existed.  ANOVA  compares  the  variance  of  sample  data 
contained  within  each  survey  group  (treatment  group)  with  the  variance  across  other 
groups  to  analyze  the  null  hypothesis  regarding  the  equity  of  the  means  for  each  group. 
ANOVA  was  conducted  between  the  three  surveys,  between  men  and  women,  and 
between  officers  and  enlisted  using  a  3  X  2  X  2  ANOVA. 

The  ANOVA  was  conducted  using  a  series  of  F- tests  to  evaluate  the  effects  of 
each  survey  on  group  means.  An  F-statistic  p-value  was  computed  and  determined 
statistically  reliable  when  the  probability  of  a  Type  I  error  was  less  than  the  set  alpha  of 
10%  (a  <  0.10).  If  the  F-statistic  p-value  was  found  to  be  larger  than  the  error  level,  the 
null  hypothesis  was  rejected  in  support  of  the  alternative  hypothesis  that  a  significant 
difference  between  means  was  found.  Any  significantly  different  findings  of  this  kind 
pointed  to  possible  response  bias  based  on  survey  administration  method. 

Summary 

This  chapter  explained  the  research  design  and  methodology  used  to  compare  the 
responses  and  response  rates  between  900  surveys  randomly  administered  to  Air  Force 
active-duty  personnel.  The  research  goal  was  to  determine  if  there  was  any  statistical 
difference  between  the  responses  or  response  rate,  and  if  that  difference  could  be 
contributed  to  bias  based  on  the  medium  of  survey  administration.  Gender  and  military 
commission  were  used  to  determine  if  any  differences  were  more  or  less  likely  for  groups 
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based  on  these  variables.  The  results  of  all  the  analysis  and  tests  will  then  be  used  to 
draw  conclusions  about  the  impact  of  the  computer  medium  on  survey  administration  in 
the  Air  Force. 
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IV.  Results  and  Analysis 


Chapter  Overview 

This  chapter  presents  the  analysis  and  findings  of  the  survey  response  data  and 
response  rates.  First,  a  review  of  the  internal  reliability  results  of  the  questionnaire  will 
be  presented.  Next,  data  analysis  will  focus  on  answering  each  the  three  research 
questions  and  associated  hypothesis  statements  proposed  in  the  previous  chapters.  This 
analysis  will  concentrate  on  response  rates  and  survey  scores  overall,  by  gender,  and  by 
military  commission.  On  completion  of  this  chapter,  each  research  question  will  be  fully 
answered. 

Questionnaire  Reliability 

Because  of  the  low  .54  internal  reliability  level  discovered  during  the  pilot  test,  it 
was  essential  to  calculate  the  Cronbach  alpha  for  the  final  data  to  verify  its  high 
reliability.  As  anticipated,  the  Cronbach  alpha  level  was  much  higher  with  a  calculated  a 
=  .89.  This  reliability  level  passed  and  exceeded  Straub’s  (1989:160)  “.80  rule-of- 
thumb”  for  a  reliable  measurement.  In  addition,  it  validated  Mowday’s  own  internal 
consistency  measurement  of  a  =  .82  -  .93  for  the  Organizational  Commitment 
Questionnaire  (OCQ)  (Mowday,  et  al.,  1979).  Thus,  the  small  alterations  to  the  original 
OCQ  to  adapt  it  to  Air  Force  personnel  did  not  change  its  reliability  in  this  study. 
Questionnaire  Results 

The  survey  consisted  of  ten  statements  to  which  respondents  agreed  or  disagreed 
according  to  a  5 -point  Likert-scale.  Although  the  actual  responses  (commitment  level 
score)  of  the  respondents  did  not  matter  to  the  objectives  of  this  study,  the  scores  were 
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used  to  determine  differences  in  responses  across  the  three  different  mediums.  As  a 
quick  overview  of  respondent  participation,  the  following  table  helps  summarize  the 
response  rates  and  demographics  of  response  data  collected.  Detailed  data  analysis, 
categorized  by  individual  question,  is  provided  in  Appendix  B. 


Table  1.  General  Demographics  and  Response  Rates. 


Paper 

Computer 

Simple 

Computer 

Complex 

Total 

Overall 

50.53% 
N=  143 

44.37% 
N=  126 

45.49% 
N=  116 

46.84% 

N  =  385 

Men 

47.52% 

N  =  67 

37.32% 
N=  53 

45.07% 
N=  64 

43.29% 
N=  184 

Women 

53.52% 
N=  76 

51.41% 

N  -  73 

46.02% 

N  =  52 

50.63% 

N  =  201 

Officer 

60.69% 

N  =  88 

46.81% 

N=66 

48.23% 

N  -  68 

51.99% 

N  =  222 

Enlisted 

39.86% 
N=  55 

41.96% 
N=  60 

42.11% 

N  =  48 

41.27% 
N=  163 

To  answer  each  research  question,  tests  for  both  response  rates  and  actual 
responses  were  conducted.  The  response  rates  were  calculated  by  taking  the  number  of 
surveys  sent,  subtracting  the  number  returned  (for  such  reasons  as  permanent  change  of 
station,  wrong  address,  retirement,  etc.)  and  dividing  this  number  into  the  number 
received  back  from  the  respondent.  For  survey  response  analysis,  the  responses  to  each 
question  for  each  person  were  combined  into  a  composite  score  of  commitment  level. 
Three  questions,  one,  six  and  ten,  were  reverse  coded  to  aid  in  reliability  and  were 
switched  to  the  corresponding  value  (1  to  5,  2  to  4,  etc.)  before  being  added  to  the 
composite  score.  Each  respondent’s  composite  score  could  range  from  a  minimum  of  10 
(very  low  commitment  level)  to  a  maximum  of  50  (very  high  commitment  level).  These 
scores  were  calculated  for  each  survey  type  and  analyzed  using  ANOVA  tests  to 
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determine  if  one  or  more  surveys  produced  statistically  significant  results  at  a  .10  alpha 
level. 

Research  Question  1  Analysis 

The  first  research  question  (RQ1)  asked:  Are  computer-based  and  paper-based 
survey  instruments  equivalent?  Two  hypothesis  statements  were  used  to  test  this 
question.  The  first  research  hypothesis  (HI a)  proposed  that  overall  means  and  variances 
would  be  statistically  different  between  computer-based  and  paper-based  surveys.  This 
hypothesis  was  based  on  research  that  indicated  that  the  computer  medium  might  affect 
survey  responses. 

Table  2  shows  the  test  results  for  hypothesis  Hla  with  the  test  (F(l,  377)-0.1 107, 
p  =  0. 74).  In  order  to  reject  the  null  hypothesis,  the  p-value  would  have  to  be  less  than 
0.10,  meaning  that  one  of  the  surveys  had  a  score  significantly  different  from  the  other 
survey.  Since  the  p-value  was  greater  than  0.10,  the  null  hypothesis  was  not  rejected.  In 
addition,  the  Welch  ANOVA  p-value  was  0.74.  Because  this  value  was  also  greater  than 
0.10,  the  evidence  for  not  rejecting  the  null  hypothesis  was  strengthened.  To  ensure  that 
constant  variance  and  normality  had  not  been  violated,  the  Levene  p-value  was  shown  to 
be  greater  than  0.10.  To  summarize  the  findings  for  hypothesis  Hla,  there  is  insufficient 
evidence  to  conclude  any  difference  between  means  and  variances  of  the  paper  and 
computer  surveys. 
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Table  2.  Test  Result  Hypothesis  HI  a,  Research  Question  One. 


|  H0:  Mean  and  Variance  Differences  =  0 

N 

Composite 

Score  Mean 

Standard 

Deviation 

Paper 

140 

38.107 

8.1502 

Computer 

Combined 

239 

38.385 

7.6637 

ANOVA 

Welch  ANOVA 
Levene 

F(l,377)=0.1 107,  p=0.74 

F(l, 276. 89)=0. 1071,  p=0. 74,  t=0.33 
F(l,377)=0.81 56,  p^0.67 

The  second  research  hypothesis  (Hlb)  proposed  that  overall  computer-based 
survey  response  rates  would  be  statistically  different  from  paper-based  response  rates. 

The  test  required  a  comparison  of  response  rates  for  the  paper  survey  with  the  combined 
response  rate  of  both  the  computer-simple  survey  and  computer-complex  survey.  Both 
computer  survey  response  rates  were  combined  since  the  comparison  was  between  paper 
and  computer  regardless  of  the  computer  survey  type.  If  it  was  found  that  the  response 
rates  were  significantly  different  between  the  paper  and  computer  versions  of  the  survey, 
the  two  versions  may  not  be  considered  equivalent  instruments. 

Table  3  presents  the  test  results  for  hypothesis  Hlb.  The  test  statistic  calculated 
by  the  inference  concerning  two  proportions  was  1 .537.  In  order  to  reject  the  null 
hypothesis,  the  test  statistic  would  have  to  be  greater  than  the  calculated  Z-score  (Zl0,2)  of 
1.645.  Since  the  test  statistic  did  not  lie  in  the  rejection  region,  there  was  insufficient 
evidence  to  reject  the  null  hypothesis.  In  other  words,  the  evidence  did  not  indicate  a 
significant  difference  in  response  rates  between  computer  and  paper  surveys.  However,  it 
is  important  to  note  that  at  higher  Z-scores  (greater  than  Z 12/2)  the  difference  would  be 
considered  statistically  significant.  Consequently,  while  the  standards  adopted  for  the 
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current  hypothesis  test  show  the  difference  to  be  statistically  insignificant,  standards  that 
are  more  liberal  would  have  found  significant  differences  between  the  two  methods  of 
administration  based  on  response  rates. 

Table  3.  Test  Result  Hypothesis  Hlb,  Research  Question  One. 


H0:  Response  Rate  Differences  =  0 

N 

Response 

Rejection 

Test 

Rate 

Region 

Statistic 

Paper 

143* 

.5053 

Z>  1.645 

Z=  1.537 

Computer 

Combined 

242* 

.4489 

1  *  indicates  different  N  from  ANOVA  tests  since  6  responses 

I  were  thrown  out  because  of  missing  or  incomplete  data  for  the 

1  ANOVA  test. 

_ 

- 1 

While  both  research  hypotheses,  Hla  and  Hlb,  failed  to  show  significant  overall 
evidence  that  paper  and  computer  mediums  elicited  different  responses  from  all  the  Air 
Force  respondents,  patterns  were  found  in  the  data  that  showed  there  might  be  slight 
effects.  For  instance,  six  of  the  seven  hypothesis  tests  to  find  differences  in  response 
rates  found  that  the  paper  survey,  on  average,  received  a  consistent  5-10%  greater 
response  rate  than  the  computer  survey.  This  significant  finding  was  hidden  by  the  actual 
hypothesis  tests.  In  other  words,  the  paper-survey  consistently  received  a  higher  response 
rate,  and  this  fact  deserves  attention. 

This  consistent  5-10%  difference  in  response  rates  is  remarkable.  Several  reasons 
can  be  provided  to  try  to  explain  the  difference.  First,  one  conclusion  is  that  Air  Force 
members  felt  more  comfortable  filling  out  a  paper  survey  versus  a  computer  survey.  One 
former  Air  Force  Institute  of  Technology  student  (Adams,  1996)  concluded  in  her 
unpublished  thesis  that  Air  Force  members  are  more  comfortable  with  written 
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correspondence  as  compared  with  e-mail  correspondence  because  the  paper  medium  is 
viewed  as  more  formal.  Another  conclusion  that  can  be  drawn  from  the  consistent  5-10% 
greater  response  rate  for  paper  surveys  is  that  respondents  were  less  comfortable  with  the 
computer  medium  rather  than  more  comfortable  with  the  paper  medium.  The  only  way 
to  know  which  viewpoint  is  more  accurate  is  to  have  a  standard  with  which  to  compare 
the  response  rates.  Unfortunately,  there  is  no  standard,  and  studies  (Sjostrom,  1999)  have 
reported  response  rates  for  the  same  population  to  be  43%  to  62%  after  a  short  time.  This 
indicates  that  one  population  may  not  have  a  standard  expected  response  rate  from  which 
to  compare  future  response  rates. 

Based  on  the  results  of  these  two  hypothesis  tests,  the  evidence  suggests  that  there 
is  no  significant  difference  between  the  paper  and  computer  surveys  and  they  can  be 
considered  equivalent  instruments.  However,  before  making  this  conclusion,  it  is 
important  to  weigh  this  result  with  the  results  of  the  next  two  research  questions.  These 
research  questions  look  at  groups  within  the  respondents.  If  one  or  more  groups  show 
differences  that  are  not  significant  enough  to  affect  the  overall  tests,  the  surveys  may  be 
found  to  be  nonequivalent  instruments  in  that  circumstance. 

Research  Question  2  Analysis 

The  second  research  question  (RQ2)  asked:  Do  complex  computer-based  surveys 
introduce  significant  bias  into  survey  responses?  Two  hypothesis  statements  were  used 
to  test  this  question.  The  first  research  hypothesis  (H2a)  proposed  that  overall  means  and 
variances  would  be  statistically  different  between  the  two  computer-based  surveys.  This 
hypothesis  was  based  on  research  that  showed  that  even  slight  format  changes  might 
affect  user  responses  indicating  bias. 


45 


Table  4  presents  the  test  results  for  hypothesis  H2a  with  the  test  (F(l, 
238)=2.3447,  p  =  0.13).  In  order  to  reject  the  null  hypothesis,  the  p-value  would  have  to 
be  less  than  0.10,  meaning  that  one  of  the  surveys  had  scores  significantly  different  from 
the  other  survey.  Since  the  p-value  was  greater  than  0.10,  the  null  hypothesis  was  not 
rejected.  In  addition,  the  Welch  ANOVA  p-value  was  0.13.  Because  this  value  was  also 
greater  than  0.10,  the  evidence  for  not  rejecting  the  null  hypothesis  was  strengthened.  To 
ensure  that  constant  variance  and  normality  had  not  been  violated,  the  Levene  p-value 
was  shown  to  be  greater  than  0.10.  To  summarize  research  hypothesis  H2a,  there  was 
insufficient  evidence  to  conclude  any  difference  between  means  and  variances  between 
the  simple  and  complex  computer  survey  types. 

Table  4.  Test  Result  Hypothesis  H2a,  Research  Question  Two. 


ANOVA  F(l,238)=2.3447,  p  =  0.13 

Welch  ANOVA  F(I,237.09)=0.2.358,  p=0.13,  t=1.536 
Levene_ F(U38)^1 467F  p=0.23 


The  next  research  hypothesis  (H2b)  proposed  that  overall  survey  response  rates 
would  be  statistically  different  between  simple  and  complex  computer-based  surveys. 
The  test  required  comparing  response  rates  for  the  paper  survey  with  the  combined 
response  rate  of  both  the  computer-simple  survey  and  computer-complex  survey.  If  it 
was  found  that  the  response  rates  were  significantly  different  between  the  two  computer 


versions  of  the  survey,  the  two  versions  may  not  be  considered  equivalent  instruments 
and  effects  of  possible  bias  would  have  to  be  considered. 

Table  5  presents  the  test  results  for  hypothesis  H2b.  The  test  statistic  calculated 
for  the  inference  concerning  two  proportions  was  0.2619.  In  order  to  reject  the  null 
hypothesis,  the  test  statistic  would  have  to  be  greater  than  the  calculated  Z-score  (Z.10/2) 
of  1 .645.  Since  the  test  statistic  did  not  lie  in  the  rejection  region,  there  was  insufficient 
evidence  to  reject  the  null  hypothesis.  In  other  words,  the  evidence  did  not  indicate  a 
significant  difference  in  response  rates  between  complex  and  simple  computer  surveys. 

Table  5.  Test  Result  Hypothesis  H2b,  Research  Question  Two. 


|  H0:  Response  Rate  Differences  =  0 

N 

Response 

Rate 

Rejection 

Region 

Test 

Statistic 

Computer- 

simple 

126* 

.4437 

Z>  1.645 

Z  =  0.2619 

| 

116 

.4549 

*  indicates  different  N  from  ANOVA  tests  since  3  responses 
were  thrown  out  because  of  missing  or  incomplete  data  for  the 
ANOVA  test. 

From  these  two  hypothesis  tests,  the  evidence  indicates  that  there  is  no  difference 
between  simple  and  complex  computer  surveys,  and  they  can  be  considered  equivalent 
instruments.  Consequently,  it  can  be  reasonably  concluded  that  significant  bias  was  not 
introduced  by  the  computer  survey  complexity.  However,  as  stated  for  research  question 
one,  further  analysis  of  groups  stratified  by  gender  and  military  commission  should  be 
considered  before  eliminating  all  uncertainty. 
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Research  Question  3  Analysis 

The  third  research  question  (RQ3)  asked:  Are  computer-based  and  paper-based 
survey  responses  or  response  rates  affected  by  a  person’s  gender  or  military  commission? 
Two  groups  of  four  research  hypotheses  were  used  to  test  this  question.  The  first  group 
of  hypotheses  focused  on  response  means  and  variances  for  men,  women,  officers  and 
enlisted  personnel.  The  second  group  of  hypotheses  focused  on  response  rates  for  men, 
women,  officers  and  enlisted  personnel.  To  determine  statistically  significant 
differences  in  response  means  between  men,  women,  officer,  and  enlisted  personnel 
across  the  three  survey  types,  a  whole  model  ANOVA  test  was  developed  and  run  using  a 
2X2X3  factorial  design.  This  test  yielded  the  results  (F(l  1,  367)  =  1.5515,  p  =  .11). 
Because  the  p-value  was  not  less  than  the  .10  significance  threshold,  this  indicated  that  no 
effect  was  statistically  significant.  In  other  words,  no  mean  differences  were  found 
among  men,  women,  officers,  and  enlisted  personnel  across  the  three  survey  types. 

Hypothesis  H3a  proposed  that  women  would  have  statistically  different  response 
means  and  variances  between  the  three  survey  types.  The  null  hypothesis,  which  stated 
that  there  was  no  difference,  was  not  rejected  in  this  case.  Similarly,  hypothesis  H4a, 
which  proposed  that  enlisted  personnel  would  have  statistically  different  response  means 
and  variances  between  the  three  survey  types  could  not  be  established.  The  null 
hypothesis  for  this  test  was  also  not  rejected. 

Both  hypotheses  H3b  and  H4b  stated  that  men  and  officers  would  have 
statistically  similar  response  means  and  variances  between  the  three  survey  types.  The 
null  hypothesis  for  both  tests  were  not  rejected  meaning  that,  as  expected,  men  and 
officers  had  statistically  similar  response  means  and  variances  across  the  three  survey 
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types.  Because  of  this  expected  result,  a  power  analysis  was  conducted  to  determine  the 
likelihood  of  a  Type  II  error.  The  power  was  calculated  to  be  .85  -  .93  for  men  and  .95  - 
.97  for  officers.  These  high  power  numbers  are  based  on  the  ability  to  detect  a 
statistically  significant  4-point  spread  in  the  means  within  groups.  Additionally,  this  high 
power  indicates  that  the  likelihood  of  having  committed  a  Type  II  error  is  low. 

The  last  four  research  hypotheses  were  examined  using  proportional  tests  of 
response  rates.  Hypothesis  H3c  proposed  that  combined  computer-based  survey  response 
rates  would  be  statistically  different  from  paper-based  response  rates  for  women. 
Hypothesis  H3d  proposed  that  the  response  rates  would  be  similar  for  men.  Hypothesis 
H4c  proposed  that  combined  computer-based  survey  response  rates  would  be  statistically 
different  from  paper-based  response  rates  for  enlisted  personnel.  Lastly,  hypothesis  H4d 
proposed  that  the  response  rates  would  be  similar  for  officers. 

Each  test  required  comparing  response  rates  for  the  paper  survey  with  the 
combined  response  rate  of  both  the  computer-simple  survey  and  computer-complex 
survey.  If  it  was  found  that  the  response  rates  were  significantly  different  between  paper 
and  computer  versions  of  the  survey,  the  two  versions  may  not  be  considered  equivalent 
instruments. 

Table  6  presents  the  test  results  for  hypotheses  H3c,  H3d,  H4c,  and  H4d,  since 
they  were  almost  identical  tests.  The  test  statistic  calculated  for  the  inference  concerning 
two  proportions  was  0.8599  for  women,  1.2382  for  men,  -0.4173  for  enlisted  personnel, 
and  2.5799  for  officers.  In  order  to  reject  the  null  hypothesis,  the  test  statistic  would  have 
to  be  greater  than  the  calculated  Z-score  (Z.IOT)  of  1 .645.  Since  the  first  three  test  statistics 
were  not  in  the  rejection  region,  there  was  insufficient  evidence  to  reject  the  null 
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hypothesis.  In  other  words,  the  evidence  did  not  indicate  a  significant  difference  in 
response  rates  between  computer  and  paper  surveys  for  men,  women,  or  enlisted 
personnel.  However,  the  test  statistic  for  officers  was  2.5799,  which  was  greater  than  the 
Z  value  of  1 .645.  Here  the  null  hypothesis  was  rejected  because  the  evidence  indicated 
there  was  a  statistically  significant  difference  between  the  way  officers  answered  the 
paper  survey  compared  with  the  computer-based  surveys. 


Table  6.  Test  Result  Hypothesis  H3c/H3d/H4c/H4d,  Research  Question  Three 


Women 


N 

Response 

Rate 

Rejection 

Region 

Test 

Statistic 

Paper 

76 

.5352 

7  1  645 

7  -  o  8599 

Computer 

Combined 

125 

.4902 

Men 

H0:  Response  Rate  Differences  =  0 

N 

Response 

Rate 

Rejection 

Region 

Test 

Statistic 

Paper 

67 

.4752 

7  >  1  645 

7  -  1  7382 

Computer 

Combined 

117 

.4120 

Enlisted 

H0:  Response  Rate  Differences  =  0 


Officers 

H0:  Response  Rate  Differences  -  0 


Paper 

Computer 

Combined 

*  Significance 


N 

Response 

Rate 

Rejection 

Region 

Test 

Statistic 

88 

.6069 

7  >  1  fsA  5 

*Z  =  2.5799 

134 

.4752 

50 


Summary 


This  chapter  presented  results  and  statistical  analysis  performed  on  data  collected 
through  the  survey  instrument  administration.  The  summarization  of  the  results  of  this 
analysis  is  presented  in  Table  7,  below.  Chapter  V  discusses  these  results  and  provides 
conclusions  as  to  what  these  results  mean  for  practitioners  and  academia  of  survey 
research. 


Table  7.  Hypotheses  Results. 


STATED  HYPOTHESIS 

RESULTS 

Overall  means  and  variances  will  be  statistically  different  between  computer- 
based  and  paper-based  surveys. 

Not  Supported 

Overall  computer-based  survey  response  rates  will  be  statistically  different 
than  paper-based  response  rates. 

Not  Supported 

Overall  means  and  variances  will  be  statistically  different  between  two 
computer-based  surveys  of  different  complexity. 

Not  Supported 

Overall  survey  response  rates  will  be  statistically  different  between  two 
computer-based  surveys  of  different  complexity. 

Not  Supported 

H3a 

Women  will  have  statistically  different  response  means  and  variances 
between  the  simple  computer,  complex  computer  and  paper-based  surveys. 

Not  Supported 

H3b 

Men  will  have  statistically  similar  response  means  and  variances  between  the 
computer-simple,  computer-complex  and  paper-based  surveys. 

Supported 

Women  will  have  a  statistically  different  response  rate  between  the  paper  and 
computer-based  surveys. 

Not  Supported 

1  H3d 

Men  will  have  a  statistically  similar  response  rate  between  the  paper  and 
computer-based  surveys. 

Supported 

■ 

Enlisted  personnel  will  have  statistically  different  response  means  and 
variances  between  the  simple  computer,  complex  computer  and  paper-based 
surveys. 

Not  Supported 

mini 

Officers  will  have  statistically  similar  response  means  and  variances  between 
the  simple  computer,  complex  computer  and  paper-based  surveys. 

Supported 

m 

Enlisted  personnel  will  have  a  statistically  different  response  rate  between  the 
paper  and  computer-based  surveys. 

Not  Supported 

H4d 

Officers  will  have  a  statistically  similar  response  rate  between  the  paper  and 
computer-based  surveys. 

Not  Supported 
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V.  Conclusions  and  Recommendations 


Chapter  Overview 

The  primary  objective  of  this  study  was  to  determine  whether  computer-based 

surveys  could  be  considered  equivalent  to  analogous  paper-based  surveys  in  a  voluntary, 

self-report  environment.  Additionally,  two  secondary  objectives  were  studied.  The  first 

was  to  determine  if  format  and  complexity  differences  among  computer-based  surveys 

introduce  significant  additional  bias  into  survey  response  rates  and  actual  responses.  The 

second  was  to  determine  if  different  survey  administration  methods  influenced  the 

response  rates  and  responses  of  various  groups.  The  three  research  questions  below  were 

developed  to  investigate  these  objectives. 

Research  Question  1.  Are  computer-based  and  paper-based  survey  instruments 
equivalent? 

Research  Question  2.  Do  complex  computer-based  surveys  introduce  significant 
bias  into  survey  responses? 

Research  Question  3.  Are  computer-based  and  paper-based  survey  responses  or 
response  rates  affected  by  a  person ’s  gender  or  military  commission? 

During  this  final  chapter,  each  of  these  questions  will  be  answered  in  relation  to  their 
associated  hypotheses  results  and  other  pertinent  information  gathered  while  analyzing 
the  respondents’  data  as  a  whole.  Following  the  answers  to  and  discussion  of  each  of 
these  research  questions,  this  study’s  limitations  will  be  discussed.  Finally,  suggestions 
for  future  research,  which  are  based  on  additional  questions  this  study  evoked,  will  be 
presented. 
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Research  Question  1 

Research  question  one  asked  if  computer-based  and  paper-based  survey 
instruments  could  be  considered  equivalent.  This  was  the  fundamental  question  of  this 
study.  As  explained  in  the  literature  review,  equivalency  in  this  study  focused  on  the 
rules  of  equivalency  as  provided  under  Classical  Test  Theory  presented  in  chapter  2.  In 
short,  this  theory  says  that  two  surveys  can  be  considered  equivalent  if  they  produce 
statistically  equal  mean  scores.  Based  on  the  results  of  the  two  hypotheses  used  to  test 
this  question  and  the  results  of  the  subsequent  tests,  it  was  determined  that  similar 
surveys  administered  via  paper  and  computer  can  be  considered  equivalent. 

Research  Question  1  Discussion 

The  results  of  this  question  indicate  that,  in  general,  a  valid  and  reliable  paper 
survey  can  be  translated  into  a  computer  survey  without  changing  the  validity  or 
reliability  of  the  survey.  However,  there  is  a  major  caveat.  This  provision  only  applies  to 
surveys  of  non-sensitive  question  content,  as  provided  in  the  current  study’s  survey. 
Surveys  of  a  sensitive  nature  may  or  may  not  be  validly  or  reliably  translated  to  a 
computer  format  because  of  the  respondent’s  greater  need  to  possibly  hide  true 
information  or  provide  false  information  as  demonstrated  in  other  key  studies  using 
sensitive  questions  (Honaker,  1988;  Lautenschlager  et  ah,  1990).  In  addition,  surveys 
that  cause  the  respondent  to  focus  on  the  survey  instrument  specifically  (e.g.  computer 
survey  about  computers)  may  introduce  validity  problems  because  the  salience  of  the 
computer  may  change  the  construct  under  study  (Webster  et  ah,  1996).  However,  a 
survey  of  organizational  or  work  related  content  should  be  reasonably  expected  to 
provide  valid  and  reliable  results  as  this  study  revealed. 
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This  study  revealed  some  interesting  trends  in  organizational  survey 
administration  that  deserve  some  attention.  As  noted  in  the  previous  chapter,  paper 
response  rate  was  consistently  higher,  around  5-10%,  for  all  groups  except  one,  enlisted 
personnel.  Although  the  difference  did  not  prove  to  be  statistically  significant,  the 
difference  appears  to  be  a  tendency  that  survey  managers  should  consider  in  survey 
design.  It  appears  that  a  paper-and-pencil  survey  will  receive  a  marginally  higher 
response  rate  compared  to  a  similar  computer-based  survey  but  not  with  all  groups.  This 
may  be  important  if  a  manager’s  goal  is  to  maximize  participation  in  an  organizational 
survey. 

The  only  group  that  did  not  have  a  higher  paper  response  rate,  enlisted  personnel, 
had  the  lowest  response  rates  of  each  survey  type.  Although  not  a  focus  of  this  survey  in 
terms  of  initial  objectives,  it  is  interesting  that  this  group  had  an  average  paper  response 
rate  of  40%  while  officers  had  an  average  paper  response  rate  of  61%.  Could  this 
difference  have  been  caused  by  the  survey  content?  Could  this  be  a  result  of  a  greater 
sense  of  duty  by  officers?  Since  the  survey  was  being  conducted  by  an  officer,  could  that 
have  caused  fellow  officers  to  participate  out  of  a  sense  of  kinship?  Data  are  not 
available  to  answer  these  questions,  but  managers  must  be  aware  that  some  groups  may 
not  want  to  answer  surveys  as  readily  as  other  groups  whether  the  medium  is  paper  or  the 
computer. 

Research  Question  2 

Research  question  two  asked  if  computer-based  surveys  of  different  format  and/or 
complexity  levels  introduce  significant  bias  into  survey  results.  Webster  et  al.  (1996) 
suggested  that  this  issue  needed  further  investigation.  She  felt  that  there  had  been  so 
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many  contradictory  findings,  when  comparing  paper  to  computer  surveys,  that  it  was 
possible  that  format  differences  were  causing  this  contradiction.  Both  Cizek  (1994)  and 
Zwick  (1990)  found  that  slight  differences  in  survey  format  and/or  question  order  could 
affect  user  responses.  Based  on  the  two  hypotheses  used  to  test  this  question,  it  can  be 
determined  that  complexity  and  format  difference  had  no  significant  effect  on  user 
responses.  Therefore,  it  is  unlikely  that  significant  bias  could  be  introduced  into  results 
based  on  complexity  or  format  differences  under  similar  circumstances. 

Research  Question  2  Discussion 

The  finding  that  complexity  and  format  differences  had  insignificant  effects  on 
non-response  and  mean  score  responses  was  not  surprising,  although  this  study 
hypothesized  the  opposite  effect.  It  was  important  to  provide  both  a  simple  computer 
survey  and  a  complex  computer  survey  in  one  study  to  enable  any  affects  to  be  studied  at 
one  time  under  uniform  circumstances.  Prior  research,  as  indicated,  was  very  concerned 
about  small  modifications  changing  the  reliability  and  even  validity  of  computer  surveys. 
This  was  even  a  suggested  as  a  cause  for  different  conclusions  being  drawn  from 
different  studies  concerning  equivalence  of  paper  and  computer  tests  and  surveys 
(Webster  et  al.,  1996). 

Over  the  past  several  decades  of  computer-based  testing  and  polling,  computer 
interfaces  have  become  easier  to  operate.  Computer  applications  are  becoming 
increasingly  intuitive,  as  programs  have  evolved  from  command  line  interpreters  to 
Windows  based  graphical  user  interfaces.  Widespread  use  of  touch  screen  and  voice 
activated  interfaces  are  the  next  evolution  in  computer  testing,  polling,  and  surveying. 
This  is  important  to  note  because  any  computer  survey  or  testing  equivalency  research 


55 


greater  than  five  to  ten  years  prior  to  the  present  day  was  done  using  interfaces  that  were 
more  difficult  to  use.  Prior  research  that  indicated  differences  between  computer-based 
surveys  or  testing  could  be  due  more  to  the  interface  presented  than  the  computer 
technology  available.  In  the  past,  a  respondent  may  have  been  required  to  physically  type 
responses  to  Likert-style  questions  or  press  strange  key  combinations  to  perform  given 
commands.  It  would  be  expected  that  format  and  complexity  differences  several  years 
ago,  would  be  more  of  a  factor  than  format  and  complexity  changes  today.  Therefore, 
past  research  that  showed  that  format  differences  affected  user  responses  was  probably  a 
greater  source  of  error  than  in  today’s  web-based  environments.  Web-based  innovations 
tend  to  help  the  user  (with  programmed  meaningful  error  messages)  rather  than  confuse 
or  dissuade  a  user  of  some  archaic  DOS-based  program.  In  other  words,  the  findings  of 
this  study  may  indicate  that  complexity  and  format  differences  may  not  be  as  significant 
between  surveys  on  the  same  medium  as  the  differences  once  were. 

Practitioners  and  academics  can  be  reasonably  assured  that  a  validated  and 
reliable  paper-based  survey  can  be  translated  into  a  computer  web-based  format  without 
having  to  make  the  computer-based  survey  look  exactly  like  the  paper-based  version. 
Furthermore,  it  is  advantageous  to  use  the  computer  environment  to  more  efficiently  lead 
a  respondent  through  a  complex  survey.  For  example,  some  paper-based  surveys  tell  the 
respondent  to  skip  section  X  if  the  answer  to  the  following  question  is  Z.  In  addition, 
some  questions  may  say  to  disregard  the  question  if  the  respondent  is  a  male.  With  a 
computer-based  survey,  the  respondent  will  only  be  provided  the  information  he  or  she 
needs  based  on  previous  inputs.  Furthermore,  data  can  be  error-checked  for  accuracy, 
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real-time,  and  provide  easy  to  understand  messages  that  require  the  respondent  to  fix  any 
erroneous  inputs. 

For  researchers,  a  computer-based  environment  allows  correct,  valid,  and  error 
free  data  to  be  saved  and  tabulated  as  it  is  received.  Statistical  results  can  be 
automatically  adjusted  with  each  respondent  with  no  additional  monitoring  by  the 
researcher.  The  more  complex  the  survey,  the  more  the  researcher  can  and  should  rely 
upon  the  programming  of  the  survey  to  take  care  of  the  collection  and  validation  of  data. 
Research  Question  3 

This  complex  question  asked  whether  a  person’s  gender  or  military  commission 
affects  the  response  he  or  she  (officer  or  enlisted)  has  towards  computer-based  and  paper- 
based  surveys.  As  gender  studies  indicated  in  the  literature  review,  men  and  women  have 
different  relationships  with  information  technology.  For  this  study,  it  was  unknown 
whether  this  relationship  (more  comfortable  for  men  and  less  comfortable  for  women) 
would  introduce  bias,  which  could  be  detected  in  response  rates  and  composite  scores.  It 
was  also  unknown  whether  bias  would  be  in  the  form  of  higher  response  rates  for  paper 
surveys  or  computer  surveys  or  higher  composite  scores  for  either  gender. 

As  the  discussion  also  indicated  in  the  literature  review,  those  with  higher 
education  tended  to  adopt  and  accept  innovations  and  information  technology  much  more 
readily  than  those  with  less  education  (Rogers,  1995).  Just  as  with  gender,  it  was 
unknown  whether  this  relationship,  theoretically  closer  for  officers  whom  all  have  college 
degrees  as  opposed  to  enlisted  members  of  whom  4.7%  have  degrees,  would  introduce 
bias,  which  could  be  detected  in  response  rates  and  composite  scores.  And  like  gender,  it 
was  also  unknown  whether  bias  would  be  in  the  form  of  higher  response  rates  for  paper 
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surveys  or  computer  surveys  or  higher  composite  scores  for  either  officers  or  enlisted 
personnel.  Based  on  the  eight  hypotheses  (H3a,  H3b,  H3c,  H3d,  H4a,  H4b,  H4c,  H4d) 
that  were  used  to  test  this  question,  it  can  be  reasonably  determined  that  military 
commission  and  gender  do  not  cause  significant  bias  to  be  introduced  into  differing 
survey  mediums  under  administration  methods  similar  to  the  ones  in  this  study. 

However,  officers  tended  to  return  paper-based  surveys  at  a  statistically  significant  higher 
rate  than  computer-based  surveys.  All  other  groups  (males,  females,  and  enlisted 
personnel)  had  statistically  similar  response  rates  across  the  various  surveys.  Even  with 
this  one  unusual  finding,  practitioners  of  surveys  for  the  military  community  can  be 
reasonably  assured  that  survey  complexity  or  format  differences  among  similar  surveys 
will  not  affect  responses,  although  response  rate  may  be  affected. 

Research  Question  3  Discussion 

A  surprising  aspect  about  this  question  is  that  military  commission  seemed  to 
factor  into  the  response  rate  oppositely  than  hypothesized  based  on  the  characteristics 
listed  by  Rogers  (1995).  Rogers  (1995)  provided  a  strong  case  that  individuals  who  had  a 
high  degree  of  leadership,  higher  aspirations,  and  a  greater  ability  to  cope  with 
uncertainty  were  more  favorable  to  adopt  innovations  such  as  computer-based  testing  or 
surveys.  This  seemed  to  indicate  that  individuals  with  these  characteristics,  namely 
officers,  would  have  been  equally  comfortable  with  the  computer  and  paper  medium. 
What  the  findings  showed  was  that  officers  tended  to  respond  less  favorably  to  the 
computer  medium,  whereas  enlisted  personnel  responded  equally  favorably  between  the 
computer  and  paper  mediums.  What  impact  does  this  have  on  survey  practitioners  and 
academia? 
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The  impact  of  this  finding  on  survey  research  depends  on  the  impact  in  relation  to 
survey  reliability  or  validity.  If  the  paper  medium  is  assumed  the  baseline  for  survey 
research,  the  question  then  becomes  how  does  a  lowered  response  rate  for  one  group  of  a 
sample  (conversely  higher  non-response  rate)  affect  computer  survey  reliability  or 
validity?  Compared  with  the  paper  response  rate  in  this  study,  the  non-response  rate  was 
different  between  the  two  mediums  by  13.2%.  Another  way  to  view  this  is  that  13.2% 
less  officers  in  the  sample  responded  to  the  computer  survey  compared  with  the  paper 
survey.  If  the  opinions  and  beliefs  of  that  13.2%  differed  significantly  from  the  rest  of 
the  respondents,  then  bias  may  have  been  a  factor  to  cause  worry.  For  this  study, 
however,  it  is  known  how  that  1 3.2%  responded  (mean  scores)  to  the  paper  survey.  They 
responded  statistically  similarly  as  the  rest  of  the  sample  because  no  difference  was  found 
in  mean  scores  between  the  two  groups.  Thus,  it  is  safe  to  assume,  in  this  case,  that  the 
higher  non-response  for  the  computer-based  survey  was  not  indicative  of  the  introduction 
of  bias.  It  can  also  be  assumed  that  reliability  and  validity  could  not  have  been  affected 
either. 

Another  point  that  the  answer  to  this  question  makes  is  that  academia’s  notions  of 
the  impact  of  technology  adoption  on  survey  response  should  be  examined.  It  was  the 
notion  of  this  study  that  greater  comfort  with  computers  meant  an  anticipated  greater  use 
of  computers.  Conversely,  the  findings  showed  that  an  assumed  high  comfort  with 
computers  did  not  correlate  with  a  greater  use  of  computers.  In  fact,  the  opposite 
occurred.  Responses  to  computer  surveys  were  the  same  or  less  than  that  of  the  paper 
survey  regardless  of  gender  or  military  commission.  The  data  actually  indicate  that  the 
paper  medium  was  regarded  higher  than  the  computer  medium  across  the  sample, 
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although  not  at  a  statistically  significant  level  except  for  the  case  of  officers.  This 
corresponds  to  Adams’  (1996)  unpublished  thesis  findings  that  written  communication 
was  seen  as  more  formal  and  task-oriented  than  the  computer  format  and  that  military 
members  had  a  higher  response  rate  with  written  media.  Practitioners  and  academia 
cannot  assume  that  a  group  that  has  a  high  acceptance  level  with  computer  technology 
will  necessarily  use  the  computer  to  a  greater  degree  compared  with  paper  within  a 
survey.  However,  this  is  not  to  say  that  a  greater  acceptance  level  with  other  computer 
applications  will  not  lead  to  greater  use  of  that  application  outside  the  context  of  surveys. 

In  summary,  the  evidence  supports  that  military  commission  has  a  slight  effect  on 
response  rate  in  a  survey  but  does  not  affect  response  means  or  variances  in  a  survey. 
Most  interestingly,  the  data  and  analysis  indicate  that  military  members  will  respond 
better  to  a  paper  survey  than  a  computer  survey  whether  they  are  officers,  enlisted 
personnel,  men,  or  women,  although  response  means  and  variances  are  not  affected. 
Lastly,  academia  and  practitioners  of  surveys  cannot  assume  that  a  greater  acceptance  of 
computer  technology  by  a  respondent  will  result  in  a  greater  use  of  computer  technology 
by  the  respondent. 

Limitations 

It  is  important  to  recognize  several  potential  limitations  in  this  study.  A  key 
limitation  was  the  use  of  the  United  States  Postal  Service  and  DoD  Official  Mail  service 
provided  by  each  base.  Delivery  of  the  survey  package  was  contingent  upon  a  good 
address  and  proper  handling  by  these  two  agencies.  Getting  the  mail  responses  back 
relied  just  as  heavily  on  a  flawless  postal  and  base  postal  system.  To  test  the  strength  of 
this  system,  the  author  picked  one  surveyed  group  and  called  each  non-respondent  in  that 
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group.  The  author  found  that  approximately  one-third  of  the  personnel  1)  never  received 
the  survey  through  official  mail  2)  moved  to  another  base  or  3)  were  on  temporary  duty  at 
another  location.  Because  all  the  surveys  were  sent  to  a  random  sample,  the  author 
believes  these  problems  were  uniform  across  all  groups.  However,  it  is  reasonable  to 
assume  that  these  problems  may  have  affected  the  results  in  some  unknown  way. 

A  second  limitation  was  the  absence  of  a  retest.  A  retest  could  have  confirmed 
the  validity  of  the  results  from  the  first  test.  The  study  could  have  benefited  by  testing 
how  response  groups  differed  over  time,  or  even  how  different  groups  responded  to 
survey  mediums  they  had  not  previously  been  given.  However,  time  limitations  for  this 
study  precluded  a  test-retest  reliability  survey. 

Another  limitation  was  the  assumption  that  officers  and  men  were  comfortable 
with  computer  technology  and  women  and  enlisted  personnel  are  not  as  comfortable  with 
computer  technology.  Although  cites  in  the  literature  review  generally  supported  these 
generalizations,  the  reality  of  these  generalizations  are  not  very  supportive.  Many 
women  and  enlisted  personnel  are  computer  experts  and  more  comfortable  with  computer 
technology  than  other  men  or  officers  in  the  Air  Force.  The  generalizations  drawn  are 
not  absolute  and  may  be  incorrect  in  many  ways.  The  survey  should  have  asked  for  a 
level  of  computer  experience  or  comfort  level  with  computers  to  better  categorize  those 
respondents  who  were  or  were  not  comfortable  with  computer  technology.  As  they 
stand,  the  generalizations  that  formed  the  hypothesis  statements  can  be  considered  weak 
at  best. 

A  final  limitation  concerns  the  complexity  level  designed  into  the  complex 
computer  based  survey.  The  complexity  level  was  very  subjective  because  no  empirical 
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data  existed  to  suggest  what  made  a  survey  simple  or  complex.  It  was  made  complex 
based  on  what  the  author  thought  complexity  entailed.  Conclusions  that  were  drawn 
based  on  differences  between  the  simple  and  complex  survey  are  difficult  to  quantify 
because  what  is  complex  for  one  person  may  not  be  complex  for  another  person.  An 
additional  post  survey  to  determine  a  level  of  complexity  from  the  respondent’s  view 
would  have  been  beneficial  to  make  sure  a  complexity  difference  actually  existed 
between  the  two  computer  surveys. 

Future  Research 

This  study,  perhaps,  generated  many  more  questions  than  it  satisfactorily 
answered.  Several  key  questions,  indicating  areas  of  future  research,  would  be  beneficial 
to  answer  for  research  groups  such  as  the  Air  Force  Survey  Branch.  First,  why  did 
officers  answer  the  paper  survey  with  such  a  high  response  rate  compared  to  any  other 
group?  It  would  be  of  interest  to  verify  that  this  is  the  case,  and  if  so,  why?  This  could 
have  implications  into  the  best  way  to  elicit  high  response  rates  from  different  groups. 

Second,  what  would  be  the  effects  of  sensitive  questions  between  paper  and 
computer  surveys?  Previous  research  shows  that  the  biggest  differences  in  responses 
normally  occurred  when  questions  were  of  a  sensitive  nature.  It  would  be  useful  to 
quantify  this  effect  for  the  Air  Force  population. 

Third,  if  given  the  choice  to  take  either  a  paper  or  a  computer  survey,  which 
survey  would  Air  Force  personnel  take?  In  this  study,  each  group  was  sent  a  specific 
survey  and  asked  to  take  it.  If  survey  medium  is  a  choice  of  a  respondent,  this  may  help 
quantify  respondent  preference.  Based  on  this  research,  it  appears  that  certain  groups, 
such  as  officers,  preferred  the  paper  medium,  whereas  male  enlisted  personnel  preferred 
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the  computer  medium.  Quantifying  these  differences  and  why  these  differences  occur 
would  certainly  add  to  this  growing  body  of  knowledge. 

Lastly,  what  is  the  relationship  between  a  person’s  likelihood  to  adopt  a  new 
technology  and  his  or  her  likelihood  to  actually  use  that  new  technology  compared  with 
the  older  technology.  This  study  found  a  conflict  between  past  research  and  actual  results 
in  the  study.  Past  research  indicated  that  the  more  educated  a  person  is  the  sooner  he  or 
she  will  adopt  the  technology.  In  practice,  this  research  seemed  to  indicate  that 
regardless  of  a  person’s  education,  he  or  she  preferred  the  old  technology  (paper)  to  the 
new  (computer).  Since  this  was  a  peripheral  issue  in  this  study,  focused  research  into  this 
area  would  be  beneficial  to  those  who  provide  innovations. 

Conclusion 

An  overall  analysis  of  the  findings  of  this  study  makes  it  reasonable  to  conclude 
that  paper-based  and  computer-based  surveys  can  be  considered  equivalent  in  a  voluntary 
self-report  environment.  Additionally,  evidence  shows  that  complexity  and  format 
differences  between  computer-based  surveys  do  not  significantly  affect  responses  or 
response  rates.  Finally,  the  data  suggests  that  it  is  improbable  that  significant  bias  was 
introduced  into  survey  results  based  on  survey  method  of  administration,  gender,  or 
military  commission. 

Some  additional  and  interesting  observations  were  made.  First,  officers  seemed 
to  be  affected  by  the  survey  medium  more  than  other  group.  In  particular,  officers 
answered  paper  surveys  at  a  statistically  significant  higher  rate  than  computer  surveys. 
Survey  practitioners  and  academia  should  understand  that  officers,  for  unknown  reasons 
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as  of  yet,  respond  better  through  the  written  medium.  Consideration  for  this  likelihood 
must  be  made,  if  a  survey  will  include  their  responses. 

A  second  observation,  similar  to  the  first  observation,  is  that  most  groups 
responded  better  (higher  response  rate)  to  the  paper  survey  as  compared  with  the 
computer  survey.  As  noted,  the  only  significant  difference  was  with  officers,  but  the  men 
and  women  groups  had  a  consistently  higher  response  rate  for  the  paper  surveys.  The 
only  group  that  did  not  have  a  higher  response  rate  for  paper  surveys  was  enlisted 
personnel.  However,  the  difference  was  a  mere  2%  meaning  that  both  computer  and 
paper  surveys  were  equally  used  with  no  apparent  prejudice  for  one  form  over  the  other. 
Practitioners  should  be  aware  that  although  there  appears  to  be  a  prejudice  overall  for  the 
paper  survey  over  the  computer  survey,  the  data  indicates  that  this  does  not  cause 
significant  bias  in  the  survey  data. 

In  summary,  the  use  of  computers  to  conduct  surveys  has  been  strengthened  by 
this  study.  There  are  still  many  questions  to  be  answered,  but  a  step  forward  has  been 
made  by  a  careful  and  well-designed  empirical  study  that  has  shown,  contrary  to  the 
expected  outcome,  that  there  is  very  little  difference  between  a  similar  paper  and 
computer  survey.  Practitioners  and  academia  can  be  more  certain  that  computer  surveys 
are  not  invalidating  the  future  of  survey  research. 
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Appendix  A:  Paper  Survey 


Personnel  Commitment  Level  Survey 


Personal  Information 


SCN  00-34 


1.  Gender:  0  Male  O  Female 

5.  Highest  Education  Level  Completed: 

2.  Rank: 

0  High  School 

0  Some  College 

0  Bachelor's  Degree 

0  Master's  Degree 

0  PhD 

3.  AFSC:  1 

4.  Major  Command:  1 

OotionaL  Personal  information  _  ...  ..  _ _  _  . 

6.  Marital  Status:  0  Single  0  Married 

10.  Household  Yearly  Gross  Income: 

7.  Number  of  Dependents: 

8.  Years  at  Residence: 

!  O  <$29,999 

0  $30,000  -  $39,999 

O  $40,000  -  $49,999 

O  $50,000  -  $59,999 

O  >$60,000 

9.  Home  of  Record: 

Survey  Instructions 


Listed  below  are  a  series  of  statements  that  represent  possible  views  that  you  might  have  about 
Air  Force  life.  Please  indicate  the  degree  of  your  agreement  or  disagreement  with  each  statement 
by  choosing  one  of  the  five  alternatives  beside  each  statement. 

1  =  Disagree,  2  =  Slightly  disagree,  3  =  Neutral,  4  =  Slightly  agree,  5  =  Agree _ 


Views  about  the  Air  Force 

«  Disagree 

1  2  3 

Agree  » 

4  5  j 

1 . 1  am  willing  to  put  in  a  great  deal  of  effort  beyond  that  normally  expected  in  order 
to  help  the  Air  Force  be  successful. 

O 

O 

o 

O 

0 

2.  1  feel  very  little  loyalty  to  the  Air  Force. 

O 

O 

0 

O 

o 

3.  1  find  that  my  values  and  those  of  the  Air  Force  are  very  similar. 

O 

O 

6 

O 

0 

4.  1  am  proud  to  tell  others  that  1  am  part  of  the  Air  Force 

O 

O 

o 

O 

0 

5.  1  am  extremely  glad  that  1  chose  the  Air  Force  over  other  work  opportunities  1  was 
considering  at  the  time  1  joined. 

O 

O 

0 

O 

0 

6.  There  is  little  to  be  gained  by  staying  in  the  Air  Force  until  retirement. 

O 

O 

0 

O 

0 

7. 1  really  care  about  the  fate  of  the  Air  Force. 

O 

O 

o 

O 

0 

8.  For  me,  this  is  the  best  of  all  possible  organizations  to  work  for. 

O 

O 

0 

. 0 . 

o 

9. 1  would  accept  almost  any  type  of  job  assignment  in  order  to  keep  working  for  the 
Air  Force. 

O 

O 

0 

0 

0 

1 0.  Deciding  to  join  the  Air  Force  was  a  definite  mistake  on  my  part. 

O 

o 

0 

. O . r 

o 
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Appendix  B:  Computer-simple  Programming  Code 


Title:  Computer  Simple  Code  (survey cs. asp) 
Name:  Capt.  Albert  E.  Franke  IV 
Date:  15  May  2000 


<%ip=  Request.ServerVariables("REMOTE_ADDR")%> 

<HTML> 

<HEAD> 

<META  http-equiv="Content-Type"  content="text/html;  charset=iso-8859-l"> 

<META  NAME=”  GENERATOR”  content="Microsoft  FrontPage  Express  2.0”> 
<TITLE>Survey</TITLE> 

</HEAD> 

<BODY  bgcolor="#ffffff!> 

<DIV  align=right><F ONT  FACE=Arial>SCN  00-34</FONT></DIV> 
<CENTER><FONT  FACE=Arial><FONT  color-#000000  > 

<STRONG>Personnel  Commitment  Level  Survey</STRONG></FONT> 

<FONT  FACE=Arial> 

<HR> 

</FONT> 

</CENTER> 

<% 

gender2  =  Request.Form(  "gender  1") 
rank2  =  Request. Form( ’’rank  1") 
afsc2  =  Request.Form(”afsclM) 
majcom2  =  Request.Form(”majcoml”) 
education2  =  Request.  Form(”educationl") 
married2  =  Request.Form(Mmarriedl") 
depends2  =  Request.Form(  "depends  1”) 
years2  =  Request.Form(  "years  1") 
city2  =  Request.Form("cityI") 
income2  =  Request.Form("incomel") 
comments  1  =  Request.Form("comments") 
ql  =  Request.  Form(?T”) 
q2  =  Request.Form("2") 
q3  =  Request.Form(”3") 
q4  =  Request.Form(”4”) 
q5  =  Request.Form(”5") 
q6  =  Request.Form("6”) 
q7  -  Request.Form(”7") 
q8  =  Request.Form("8”) 
q9  =  Request.Form("9") 
qlO  =  Request.Form("10") 

if  gender2  -  OR  rank2  =  ""  OR  afsc2  =  ""  OR  majcom2  =  ""  OR  education  =  ”"  then 

%> 

<CENTER><FONT  FACE=Arial>Personal  Information<HR> 

</FONTx/CENTER> 

<FORM  ACTION- ’surveycs.asp”  method="post”> 

<CENTER> 

<TABLE  border=0> 

<TR> 
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<TD  height=15  style="HEIGHT:  15px"xFONT  FACE=Arial>l.  Gender:  <INPUT 
NAME=genderl  type=radio  VALUE=Male>&nbsp;Male&nbsp;&nbsp;  <INPUT  NAME=genderl 
type=radio  VALUE=Female>&nbsp;Female&nbsp;&nbsp;  </FONT></TD> 

<TD><FONT  FACE=Arial>&nbsp;&nbsp;&nbsp;  </FONT></TD> 

<TDxFONT  FACE=Arial>&nbsp;&nbsp;&nbsp;  </FONT></TD> 

<TD><FONT  FACE=Arial>&nbsp;&nbsp;  </FONT></TD> 

<TD  VALIGN=top><F ONT  FACE=Arial>&nbsp;5.  Highest 
Education  Level  Completed:&nbsp;  </FONT> 

</TD> 

</TR> 

<TRxTD> 

<P><FONT  FACE=Arial>2.  Rank:  <INPUT  NAME=rankl  size=10>  </FONT>  </P> 

<P><FONT  FACE=Arial>3.  AFSC:  <INPUT  NAME=afscl  size=l  1>  </FONT>  </Pxp><FONT 
FACE=Arial>4.  Major  Command:  <INPUT  NAME=majcoml 
size=l  1  ></FONTx/P> 

</TD> 

<TD><FONT  FACE=Arial>&nbsp;</FONT></TD><TD><FONT  FACE=Arial>&nbsp; 
</FONT> 

<P>&nbsp;</P></TD><TDxFONT  FACE=Arial>&nbsp;</FONT></TD> 

<TD><FONT  FACE=Arial><INPUT  TYPE="radio"  NAME=educationl  VALUE="HS"> 
High  School<BR> 

<INPUT  TYPE="radio"  NAME=educationl  VALUE="SC"> 

Some  College<BR> 

<INPUT  TYPE="radio"  NAME=educationl  VALUE="B"> 

Bachelor's  Degree<BR> 

<INPUT  TYPE="radio"  NAME=educationl  VALUE="M"> 

Master's  Degree<BR> 

<INPUT  TYPE="radio"  NAME=educationl  VALUE="P"> 

PhD</FONT> 

</TD> 

</TR> 

</TABLE> 

</CENTER><F  ONT  FACE=Arial> 

<HR> 

</FONT> 

<CENTER> 

<FONT  FACE=Arial>Optional  Personal  information</FONT> 

<HR> 

<DIV  align=left>&nbsp;</DIV> 

<TABLE  border=0  style="HEIGHT :  154px;  WIDTH:  700px"  id=TABLEl> 

<TR> 

<TD  height=15  style="HEIGHT :  15px"xFONT 
FACE=Arial>6.  Marital  Status:  <INPUT  NAME=marriedl 

type=radio  VALUE=Single>&nbsp;Single&nbsp;&nbsp;  <INPUT  NAME=marriedl 
type=radio  VALUE=Married>&nbsp;Married 
</FONT>  </TD> 

<TDXF0NT  FACE=Arial>&nbsp;&nbsp;&nbsp;  </FONT>  </TD> 

<TDxFONT  F ACE= Arial>&nbsp ; &nbsp ; &nbsp ;  </FONT>  </TD> 

<TDxFONT  FACE=Arial>&nbsp;&nbsp;  </FONT>  </TD> 

<TD  VALIGN=topxFONT  FACE=Arial>&nbsp;10.  Household  Yearly  Gross  Income: 

</FONT> 

</TDx/TR> 

<TR> 

<TD> 
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<P><FONT  FACE-Arial>7.  Number  of  Dependents:  <INPUT  NAME-dependsl  size-10>  </FONT> 

</P> 

<P><FONT  FACE=Arial>8.  Years  at  Residence:  <INPUT  NAME-yearsl  size-1  1>  </FONT>  </P> 
<P><FONT  FACE=Arial>9.  Home  of  Record:  <INPUT  NAME-cityl  size-1  1  ></FONT></P></TD> 
<TD><FONT  FACE=Arial>&nbsp;</FONT></TD> 

<TD><FONT  F  A  CE- Arial>&nb  sp ;  </FONT> 

<P>&nbsp;</P></TD> 

<TD><FONT  FACE=Arial>&nbsp  ;</FONT></TD> 

<TD> 

<P><FONT  FACE-ArialxINPUT 

NAME-income  1  type-radio  VALUE=30K>  &nbsp;&lt;  $29,999<BR><INPUT 
NAME-income  1  type-radio  VALUE-40K>  $30,000  -  $39,999<BR><INPUT 
NAME=incomel  type-radio  VALUE-50K>  $40,000  -  $49,999<BR><INPUT 
NAME-income  1  type-radio  VALUE-60K>  $50,000  -  $59,999<BR><INPUT 
NAME-incomel  type-radio  VALUE-60KPlus>  &gt; 

$60,000</FONT>  </P></TDx/TR> 

</TABLE> 

<HR> 

Survey  Instructions 

</CENTER 

<HR> 

<CENTERxFONT  FACE=Arial>  Listed  below  are  a  series  of  statements  that  represent  possible  views 
that  you  might  have  about  Air  Force  life.  Please  indicate  the  degree  of  your  agreement  or  disagreement 
with  each  statement  by  choosing  one  of  the  five  alternatives  beside  each  statement. 

<BR>1  =  Disagree,  2  =  Slightly  disagree,  3  -  Neutral,  4  =  Slightly  agree,  5  -  Agree</CENTER> 

<FONT  FACE-Arial> 

<HR> 

</FONT> 

<CENTERXTABLE  border-Mln> 

<TR> 

<TD  VALIGN="top”  width="500”xp 

align-”center"xFONT  FA CE-ArialXstrong> Views  about  the  Air 
Force</strongx/FONTx/P> 

</TD> 

<TD  VALIGN=”bottom”  width-”200"> 

<DIV  align-leftxFONT  FACE-ArialxFONT  size-2 xSTRONGxFONT 
size=3>&nbsp;&lt;&lt; 

Disagree&nbsp;&nbsp;&nbsp;&nbsp;  Agree  &gt;&gt;</FONT> 
</STRONG>&nbsp;<BRx/FONT>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;l&nbsp;&nbsp;&nbsp;&nbsp;&n 
bsp;&nbsp;2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs 
p;&nbsp;4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;5 
</FONT></DIV> 

</TD> 

</TR> 

<TR> 

<td  width-” 5 00"xF  ONT  FACE=Arial> 

1 . 1  am  willing  to  put  in  a  great  deal  of  effort  beyond  that  normally  expected  in  order  to  help  the  Air 
Force  be  successful.</FONT> 

</TD> 

<td  width=”200”xFONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME-1  type-radio 
VALUE=l>&nbsp;&nbsp;&nbsp;<INPUT  NAME-1  type-radio 
VALUE=2>&nbsp;&nbsp;&nbsp;&nbsp;<lNPUT  NAME-1 

type-radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME-1  type-radio 
VALUE=4>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME-1  type-radio  VALUE=5x/FONTx/TD> 
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</TR> 

<TR> 

<td  width="500"xFONT  FACE=Arial> 

2. 1  feel  very  little  loyalty  to  the  Air  Force.  </FONT> 

</TD> 

<td  width="200"xFONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=2  type=radio  VALUE=l>&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=2  type=radio  VALUE=2>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=2  type=radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=2  type=radio  VALUE=4>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=2  type=radio  VALUE=5> 

</FONT></TD> 

</TR> 

<TR> 

<td  width="500"><FONT  FACE=Arial> 

3. 1  find  that  my  values  and  those  of  the  Air  Force  are  very  similar.</FONT> 

</TD> 

<td  width="200"><FONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUTNAME=3  type=radio  VALUE=l>&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=3  type=radio  VALUE=2>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=3  type=radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=3  type=radio  VALUE=4>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=3  type=radio  VALUE=5> 

</FONT></TD> 

</TR> 

<TR> 

<td  width="500"><FONT  FACE=Arial> 

4. 1  am  proud  to  tell  others  that  I  am  part  of  the  Air  Force</FONT> 

</TD> 

<td  width="200"xFONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=4  type^radio  VALUE=l>&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=4  type=radio  VALUE=2>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=4  type=radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=4  type=radio  VALUE=4>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=4  type=radio  VALUE=5></FONT></TD> 

</TR> 

<TR> 

<td  width="500"><FONT  FACE=Arial> 

5. 1  am  extremely  glad  that  I  chose  the  Air  Force  over  other  work  opportunities  I  was  considering 
at  the  time  I  joined. </FONT> 

</TD> 

<td  width="200"xFONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=5  type=radio  VALUE=l>&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=5  type=radio  VALUE=2>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=5  type=radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=5  type=radio  VALUE=4>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=5  type=radio  VALUE=5></FONTx/TD> 

</TR> 

<TR> 

<td  width="500"xFONT  FACE=Arial> 

6.  There  is  little  to  be  gained  by  staying  in  the  Air  Force  until  retirement.  </FONT> 

</TD> 

<td  width="200"xFONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=6  type=radio  VALUE=l>&nbsp;&nbsp;&nbsp; 
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<INPUT  NAME=6  type=radio  VALUE=2>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=6  type=radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=6  type=radio  VALUE=4>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=6  type=radio  VALUE=5> 

</FONTx/TD> 

</TR> 

<TR> 

<td  width="500"><FONT  FACE=Arial> 

7. 1  really  care  about  the  fate  of  the  Air  Force. </FONT> 

</TD> 

<td  width="200"><FONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=7  type=radio  VALUE=l>&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=7  type=radio  VALUE=2>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=7  type=radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=7  type=radio  VALUE=4>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=7  type=radio  VALUE=5></FONT></TD> 

</TR> 

<TR> 

<td  width="500"><FONT  FACE=Arial>8.  For  me,  this  is  the 
best  of  all  possible  organizations  to  work  for.</FONT> 

</TD> 

<td  width="200"xFONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=8  type=radio  V A L U E  - 1  >& nb s p ; & n b s p ; & n h s p ; 

<INPUT  NAME=8  type=radio  VALUE=2>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=8  type  radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=8  type=radio  VALUE=4>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=8  type=radio  VALUE=5x/FONTx/TD> 

</TR> 

<TR> 

<td  width="500"><FONT  FACE=Arial> 

9. 1  would  accept  almost  any  type  of  job  assignment  in  order  to  keep  working  for  the  Air 
Force. </FONT> 

</TD> 

<td  width="200"><FONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=9  type=radio  VALUE=l>&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=9  type=radio  VALUE=2>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=9  type=radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=9  type=radio  VALUE=4>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=9  type=radio  VALUE=5></FONTx/TD> 

</TR> 

<TR> 

<td  width="500"xFONT  FACE=Arial> 

10.  Deciding  to  join  the  Air  Force  was  a  definite  mistake  on  my  part.  </FONT> 

</TD> 

<td  width="200"xFONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp; 
<INPUTNAME=10  type=radio  VALUE=l>&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=10  type=radio  VALUE=2>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=10  type=radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp; 
<INPUTNAME=10  type=radio  VALUE=4>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=10  type=radio  VALUE=5x/FONTx/TD> 

</TR> 

</TABLE> 

</CENTER> 

<CENTER>&nbsp  ;</CENTER> 


70 


<DIV  align=leftxFONT  FACE^Arial>Do  you  have  any  additional  Comments? 

</F  ONT></DI  V  > 

<DIV  align^centerxTEXTAREA  id^comments  NAME^comments  style- 'HEIGHT:  62px;  WIDTH: 
664px"  x/TEXT  ARE  Ax/DI  V  > 

<p  align="c  enter  "> 

<FONT  FACE-ArialxINPUT  TYPE="submif '  VALUE=" Submit  answers'^ 

</FONT>  </P></TD></TR> 

</TABLE> 

</form> 

<%  else 

msg  body  =  "Date:  "  &  date  &  chr(13)  &  "Time:  "  &  time  &  chr(13)  &  "IP:  "  &  ip  &  chr(13)  &  "Gender:  " 
&  gender2  &  chr(13)  &  "Rank:  "  &  rank2  &  chr(13)  &  "AFSC:  "  &  afsc2  &  chr(13)  &  "MAJCOM:  "  & 
majcom2  &  chr(13)  &  "Education:  "  &  education2  &  chr(13)  &  "Married: "  &  married2  &  chr(13)  & 
"Dependents:  "  &  depends2  &  chr(13)  &  "Years:  "  &  years2  &  chr(13)  &  "City:  "  &  city2  &  chr(13)  & 
"Income:  "  &  income2  &  chr(13)  &  "1.  "  &  ql  &  chr(13)  &  "2.  "  &  q2  &  chr(13)  &  "3.  "  &  q3  &  chr(13)  & 
"4.  "  &  q4  &  chr(13)  &  "5. "  &  q5  &  chr(13)  &  "6.  "  &  q6  &  chr(13)  &  "7.  "  &  q7  &  chr(13)  &  "8.  "  &  q8 
&  chr(13)  &  "9.  "  &  q9  &  chr(13)  &  "10.  "  &  qlO  &  chr(13)  &  "comments:  "  &  comments  1  &  chr(13) 

’  mail  file 

set  objMail  =  Server.CreateObject("CDONTS.NewMail") 

objMail.From  =  "albert.franke@afit.af.mil" 

objMail.To  =  "albert.franke@afit.af.mil" 

objMail. Subject  =  "Computer  Survey  Simple  Results" 

objMail.Body  =  msgbody 

objMail.  SendQ 

set  objMail  =  nothing 

’  Place  information  into  database 

Set  OBJdbConnection  =  Server.CreateObject("ADODB.Connection") 

OBJdbConnection.Open  "survey" 

string  1="INSERT  INTO  survey  VALUES  (" 

sql  =  string  1  &  &  date  &  ",,,n  &  time  &  "7”  &  gender2  &  m,m  &  rank2  &  "7"  &  afsc2  &  & 

majcom2  &  &  education2  &  &  married2  &  ”7"  &  depends2  &  "V”  &  years2  &  "7"  &  city2  &  ",,m 

&  income2  &  "7"  &  ql  &  "7"  &  q2  &  "7"  &  q3  &  "7"  &  q4  &  "7"  &  q5  &  "7"  &  q6  &  M7M  &  q7  &  "7"  & 
q8  &  "7"  &  q9  &  &  qlO  &  "7"  &  ip  &  T 

OBJdbConnection.Execute(sql) 

OBJdbConnection.close 

%xfont  color="#ffD000"  size="4"  x/font> 

<P  align=centerxfont  color="#ff0000"  size- '4"  > 

</fontxFONT  FACE=Arial>&nbsp;</FONT></P> 

<P  align=centerxF ONT  color=#ffD000  FACE="Arial"  size  =4>Data  sent!&nbsp;</FONTx/P> 

<P  align=centerxFONT  color=#ff0000  size=4  > 

<FONT  FACE=Arial>Thank  you  for  submitting  your  views  on  the  Air  Force. 
<BR></FONT></Px/FONT> 

<%  end  if  %> 

</FONT> 

</BODY> 

</HTML> 
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Appendix  C:  Computer-complex  Programming  Code 


Title:  Computer  Complex  Code  (surveycc.asp) 
Name:  Capt.  Albert  E.  Franke  IV 
Date:  15  May  2000 


<%ip=Request.ServerVariables("REMOTE_ADDR") 
pagel  =  Request.Form("page") 
if  pagel  =  ""  then 
pagel  = "1" 
end  if 
%> 

<HTML> 

<HEAD> 

<TITLE>Survey</TITLE> 

</HEAD> 

<BODY  bgcolor="#20b2aa"> 

<DIV  align=right><font  color="#000000"  size="3"  FACE=Arial> 

SCN  00-34</font></DIV> 

<CENTER><FONT  FACE=ArialxF  ONT  color=#000000  size=3> 

<STRONG>Personnel  Commitment  Level  Survey</STRONGx/FONT>  </FONT>  <FONT 
FACE=Arial> 

<HR> 

</FONT> 

<FORM  NAME  =  "forml"  ACTION  =  "surveycc.asp"  METHOD  =  post> 

<% 

select  case  pagel 
case  "1" 

%> 

</CENTER><CENTER><FONT  FACE=Arial>Personal  Information  (page  1  of  3)<HR> 
</FONTx/CENTER> 

<CENTER> 

<INPUT  NAME="page"  VALUE="2"  type=hidden> 

<TABLE  border=0> 

<TR> 

<tdheight=15  style="HEIGHT:  15px”xFONT  FACE=Arial>l.  Gender: 

<INPUT  TYPE="radio”  NAME="genderl "  VALUE="Male"> 
&nbsp;Male&nbsp;&nbsp; 

<INPUT  TYPE="radio"  NAME="genderl"  VALUE="Female">&nbsp;Female 
&nbsp;&nbsp; 

</FONT> 

</TD> 

<TDxFONT  FACE=Arial>&nbsp;&nbsp;&nbsp;  </FONT> 

</TD> 

<TDxFONT  FACE=Arial>&nbsp;&nbsp;&nbsp;  </FONT> 

</TD> 

<TDxFONT  F ACE= Arial>&nbsp ; &nbsp ;  </FONT> 

</TD> 

<TD  VALIGN=topxFONT  FACE=Arial>&nbsp;5.  Highest 
Education  Level  Completed:&nbsp;  </FONT> 

</TD> 

</TR> 

<TR> 
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<TD> 

<P><FONT  FACE=Arial>2.  Rank: 

<SELECT  id=rankl  NAME=rankl>  <OPTION  selected 
VALUE="None  Selected">Select  One</OPTION><OPTION 
VALUE=AB>Airman  Basic</OPTION><OPTION  VALUE=Amn> 

Airman</OPTION><OPTION  VALUE=AlC>Airman  First  Class 

</OPTION><OPTION  VALUE=SrA>Senior  Airman</OPTIONxOPTION  VALUE=SSgt>Staff 
Sergeant</OPTION><OPT!ON  VALUE=TSgt>Technical  Sergeant</OPTION><OPTION 
VALUE=MSgt>  Master  Sergeant</OPTION><OPTION  VALUE=SMSGT>Senior  Master 
Sergeant</OPTION><OPTION  VALUE=CMSGT 

>Chief  Master  Sergeant</OPTION><OPTION  VALUE=2Lt>Second  Lieutenant 
</OPTION><OPTION  VALUE=lLt>First  Lieutenant</OPTION><OPTION 
VALUE=Capt>Captain</OPTION><OPTION  VALUE=Maj>  major</OPTION>  <OPTION 
VALUE=LtCol>Lieutenant  Colonel</OPTION><OPTION  VALUE=  Col>Colonel 
</OPTION></SELECT>&nbsp;  </FONT>  </P> 

<PxFONT  FACE=Arial>3.  AFSC:  <INPUT  NAME=afscl  size=ll>  </FONT>  </P> 

<PxFONT  FACE=Arial>4.  Major 

Command:  <SELECT  id=majcoml  NAME=majcoml>  <OPTION  selected 
VALUE="No  Selection">Select  One</OPTIONxOPTION 

VALUE=ACC>ACC</OPTIONxOPTION  VALUE=AETC>AETC</OPTIONxOPTION 
VALUE=AFMC>AFMC</OPTIONxOPTION  VALUE=AFSOC>AFSOC</OPTIONxOPTION 
VALUE=AFSPC>AFSPC</OPTIONxOPTION  VALUE=AMC>AMC</OPTIONxOPTION 
VALUE=PACAF>PACAF</OPTIONxOPTION  VALUE=USAFE>USAFE</OPTIONxOPTION 
VALUE=Other>Other</OPTIONx/SELECTx/FONT></P> 

</TD> 

<TDxFONT  FACE=Arial>&nbsp;</FONTx/TD><TDxFONT  FACE=Arial>&nbsp; 
</FONT> 

<P>&nbsp;</Px/TD><TDxFONTFACE=Arial>&nbsp;</FONTx/TD> 

<TD> 

<P><FONT  FACE=ArialxINPUT  TYPE="radio"  NAME=educationl  VALUE="HS"> 

High  School<BR> 

<INPUT  TYPE-'radio"  NAME=educationl  VALUE="SC"> 

Some  College<BR> 

<INPUT  TYPE-'radio"  NAME=educationl  VALUE="B"> 

Bachelor's  Degree<BR> 

<INPUT  TYPE="radio"  NAME=educationl  VALUE="M"> 

Master's  Degree<BR> 

<INPUT  TYPE="radio"  NAME=educationl  VALUE=”P">  PhD</FONT>  </P> 
<P>&nbsp;</P> 

</TD> 

</TR> 

</TABLE> 

</CENTER> 

<CENTERxINPUT  TYPE  =  "submit"  VALUE  =  "Next  Page  >"  NAME  =  "Submit"x/CENTER> 
<CENTER>&nbsp  ;</CENTER> 

<SCRIPT  LANGUAGE="VBSCRIPT"> 

<!- 

Function  SubmitOnClick 
Dim  myForm 

Set  myF  orm  =  document,  form  1 
Submit_OnCl  i  ck=True 

If  (Not((myForm.genderl(0).checked)  or  (myForm.genderl(l).checked)))  Then 
MsgBox  "Please  select  a  gender  in  Question  1  ",0, "Missing  Information" 

SubmitOnClick  =  False 
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End  If 

If  ( my  Form,  rank  1. value)  =  "None  Selected"  Then 
MsgBox  "Please  select  a  rank  in  Question  2",0, "Missing  Information" 

SubmitOnClick  =  False 
End  If 

If  (myForm.afsc  Lvalue)  =  "*'  Then 

MsgBox  ’’Please  enter  an  AFSC  in  Question  3 ”,0, ’’Missing  Information” 

SubmitOnClick  =  False 
End  If 

If  (myForm.majcoml  .value)  -  "No  Selection”  Then 
MsgBox  "Please  select  a  MAJCOM  in  Question  4", 0, "Missing  Information” 

Sub  mitOnC  lick  —  False 
End  If 

If  (Not((myForm.educationl(0).checked)  or  (my Form. educationl(l). checked)  or 
(myForm.educationl(2).checked)  or  (myForm.educationl(3).checked)  or 
(myForm. education  1  (4). checked)))  Then 
MsgBox  "Please  select  an  education  level  in  Question  5”, 0, "Missing  Information" 

SubmitOnClick  =  False 
End  If 

End  Function 
— > 

</SCRIPT> 

</FORM> 

<% 

case  "2" 

gender2  =  Reque  st.Form("  gender  1") 
rank2  =  Request.Form("rankl") 
afsc2  =  Request.Form("afscl") 
majcom2  =  Request.Form("majcoml") 
education2  =  Request.Form("educationl")%> 

<FONT  FACE-Arial> 

</FONT> 

<CENTER><FONT  FACE=Arial>Optional  Personal  Information  (page  2  of 
3)</FONT></CENTER><FONT  FACE=Arial> 

<CENTER> 

<HR> 

</FONT> 

<INPUT  NAME-"page"  VALUE-"3"  type-hidden> 

<TABLE  border-0  style="HEIGHT:  174px;  WIDTH:  581px"> 

<TR> 

<TD  height-15  style="HEIGHT:  15px"> 

<P  align=left><FONT  FACE-Arial>6.  Marital  Status:  <INPUT  NAME-marriedl 
type-radio  VALUE=Single>&nbsp;Single&nbsp;&nbsp;  <INPUT  NAME-marriedl 
type-radio  VALUE-Married>&nbsp;Married</FONT></P>  </TD> 

<TD  VALIGN-top> 

<P  align-left><FONT  FACE=Arial>10.  Household  Yearly  Gross  Income: 

</FONTx/p> 

</TDx/TR> 

<TR> 

<TD> 

<P  align-leftxFONT  FACE-Arial>7.  Number  of  Dependents:  <INPUT  NAME-depends  1  size-10> 
</FONTx/p> 

<P  align-leftxFONT  FACE=Arial>8.  Years  at  Residence:  <INPUT  NAME-yearsl  size=l  1> 

</FONT></P> 
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<P  align=left><FONT  FACE=Arial>9.  Home  of  Record:  <INPUT  NAME-cityl 
size=l  lx/FONTx/P></TD> 

<TD> 

<FONT  FACE=Arial> 

<INPUTNAME=mcomel  type=radio  VALUE=30K>&lt;  $29,999<BR> 

<INPUT  NAME=incomel  type=radio  VALUE=40K>  $30,000  -  $39,999<BR> 
<INPUT  NAME=incomel  type=radio  VALUE=50K>  $40,000  -  $49,999<BR> 
<INPUT  NAME=incomel  type=radio  VALUE=60K>  $50,000  -  $59,999<BR> 
<INPUT  NAME=incomel  type=radio  VALUE=60KPlus>  & gt;  $60,000</FONT> 
<P>&nbsp ;</P></TDx/TRx/TABLE><FONT  FACE=Arialx/FONT> 
<INPUT NAME="gender  1"  VALUE="<%=gender2%>"  type=hidden> 

<INPUT  NAME-'rankl"  VALUE="<%=rank2%>M  type=hidden> 

<INPUT NAME-'afscl"  VALUE="<%==afsc2%>"  type=hidden> 

<INPUT  NAME- 'majcoml  *'  VALUE="<%=majcom2%>"  type=hidden> 

<INPUT  NAME^” education  1  "  VALUE="<%=education2%>"  type=hidden> 
<INPUT  id=submit  NAME=submit  type^submit  VALUE- 'Next  Page  >"> 
&nbsp;<FONT  FACE=ArialXp> 

<SCRIPT  LANGUAGE="VBSCRIPT”> 

<!-- 

Function  SubmitOnClick 
Dim  my  Form 

Set  my  Form  =  document,  form  1 
SubmitOnClick^True 
If  Len(myForm.depends  Lvalue)  >  0  Then 
If  (Not(IsNumeric(myForm. depends  Lvalue)))  Then 
MsgBox  "Please  only  use  numeric  values  for  Question  7", 0, "Missing  Information" 
SubmitOnClick  =  False 
End  If 
End  If 

If  Len(myForm.yearsl.value)  >  0  Then 
If  (Not(IsNumeric(myForm. years  Lvalue)))  Then 
MsgBox  "Please  only  use  numeric  values  for  Question  8",0, "Missing  Information" 
SubmitOnClick  =  False 
End  If 
End  If 

End  Function 
— > 

</SCRIPT> 

</FORM> 

<%case  "3" 

married2  =  Request. Form(  "married  1") 
depends2  =  Request.  Form(  "depends  1") 
years2  =  Request.Form("yearsl") 
city2  =  Request. Form("cityl") 
income2  =  Request.  Form("  income  1") 
gender2  =  Request.  Form("  gender  1") 
rank2  =  Request.Form("rankl") 
afsc2  =  Request.Form("afscl") 
majcom2  =  Request.Form("majcoml") 
education2  =  Request.Form("educationl") 

%> 

</FONT> 

<INPUT  NAME="page"  VALUE="4"  type=hidden> 

<pxFONT  FACE=Arial>Survey  Instructions  (page  3  of  3)</FONT>  </CENTER> 
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<FONT  FACE=Arial> 

<HR> 

</FONT> 

<CENTER><FONT  FACE=Arial>  Listed  below 

are  a  series  of  statements  that  represent  possible  views  that  you  might  have 
about&nbsp;Air  Force  life.  Please  indicate  the  degree  of  your  agreement  or 
disagreement  with  each  statement  by  choosing  one  of  the  five  alternatives 
beside  each  statement  <BR> 

1  =  Disagree, &nbsp;2  =  Slightly  disagree, &nbsp;3  =  Neutral, &nbsp;4  =  Slightly  agree, &nbsp;5  =  Agree 
</CENTER><FONT  FACE=Arial> 

<HR> 

</FONT> 

<CENTER><T  AB  LE  border='T’> 

<TR> 

<TD  VALIGN-"top"  width="500"xp 

align-'center"xFONT  FACE=Arial><strong> Views  about  the  Air 
F  ore  e</strong></F  ONTX/P> 

</TD> 

<TD  VALIGN- ’bottom"  width="200"> 

<CENTERxSTRONG>&lt;&lt; 

Disagree&nbsp;&nbsp;&nbsp;  Agree 

&gt;&gt;</STRONGxBR>&nbsp;l&nbsp;&nbsp;&nbsp;<&nbsp;&nbsp;&nbsp;&nbsp;2&nbsp;&nbsp;& 
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;4&nbsp;&nbsp;&n 
bsp ;  &nb  sp ;  &nbsp ;  &nbsp ;  &nb  sp ;  5  </  CENTER> 

</TD> 

</TR> 

<TR> 

<td  width- ’500"xFONT  FACE=Arial>l.&nbsp;  I  am  willing  to 
put  in  a  great  deal  of  effort  beyond  that  normally  expected  in  order  to 
help  the  Air  Force  be  successful. </FONT> 

</TD> 

<td  width- '200"xFONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=al  type=radio  VALUE=l>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=al  type=radio  VALUE=2>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=al  type^radio  VALUE-3>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=al  type=radio  VALUE=4>&nbsp;&nbsp;&nbsp;&nbsp; 

<INPUT  NAME=al  type=radio  VALUE=5x/FONTx/TD> 

</TR> 

<TR> 

<td  width- ,500"xFONT  FACE=Arial>2.&nbsp;  I  feel  very 
little  loyalty  to  the  Air  Force.  </FONT> 

</TD> 

<td  width="200"xFONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a2 

type=radio 

VALUE^^&nbsp^nbspj&nbspj&nbsp^INPUT  NAME=a2  type^adio 
VALUE^2>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a2 

type=radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a2  type=radio 
VALUE=4>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a2  type=radio  VALUE=5x/FONTx/TD> 

</TR> 

<TR> 

<td  width- ’500"xFONT  FACE=Arial>3.&nbsp;  I  find  that  my 
values  and  those  of  the  Air  Force  are  very  similar.</FONT> 

</TD> 
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<td  width="200"xFONT  FACE-Arial>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME-a3 

type-radio 

VALUE-  1  >&nbsp  ;&nbsp ; &nbsp  ;&nbsp ;<INPUT  NAME-a3  type-radio 
VALUE-2>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a3 

type-radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME-a3  type-radio 
VALUE-4>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME-a3  type-radio  VALUE=5></FONT></TD> 
</TR> 

<TR> 

<td  width-" 5 00 "xF ONT  FACE=Arial>4.&nbsp;  I  am  proud  to 
tell  others  that  I  am  part  of  the  Air  Force</FONT> 

</TD> 

<td  width-"200"><FONT  FACE-Arial>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME-a4 

type-radio 

VALUE- 1  >&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a4  type-radio 
VALUE-2>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a4 

type-radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a4  type-radio 
V ALUE-4>&nbsp ;&nbsp ;&nbsp ;&nbsp ;<INPUT  NAME-a4  type-radio  VALUE-5></FONT></TD> 
</TR> 

<TR> 

<td  width-"500"><FONT  FACE-Arial>5.  I  am  extremely  glad 
that  I  chose  the  Air  Force  over  other  work  opportunities  I  was  considering 
at  the  time  I  joined.</FONT> 

</TD> 

<td  width-" 200 "><FONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME-a5 

type-radio 

VALUE-l>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a5  type-radio 
VALUE-2>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a5 

type-radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME-a5  type-radio 
VALUE-4>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a5  type-radio  VALUE— 5></FONTx/TD> 
</TR> 

<TR> 

<td  width-"500"xFONT  FACE=Arial>6.&nbsp;  There  is  little 
to  be  gained  by  staying  in  the  Air  Force  until  retirement.  </FONT> 

</TD> 

<td  width="200"xFONT  FACE-Arial>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a6 

type-radio 

VALUE- l>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME-a6  type-radio 
VALUE-2>&nbsp;&nbsp;&nbsp;&nbsp;<INPUTNAME-a6 

type-radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME-a6  type-radio 
VALUE-4>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a6  type-radio  VALUE-5 x/FONTx/TD> 

</TR> 

<TR> 

<td  width-" 500"xFONT  FACE-Arial>7.  I  really  care  about 
the  fate  of  the  Air  Force.</FONT> 

</TD> 

<td  width-"200"xFONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME-a7 

type— radio 

VALUE-l>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a7  type-radio 
VALUE-2>&nbsp;&nbsp;&nbsp;&nbsp;<INPUTNAME-a7 

type-radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp;<FISrPUT  NAME-a7  type-radio 
VALUE-4>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME— a7  type-radio  VALUE-5x/FONTx/TD> 

</TR> 

<TR> 

<td  width="500"xFONT  FACE-Arial>8.&nbsp;  For  me,  this  is 
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the  best  of  all  possible  organizations  to  work  for.</FONT> 

</TD> 

<td  width="200"><FONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a8 

type=radio 

VALUE=l>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a8  type=radio 
VALUE=2>&nbsp;&nbsp;&nbsp;&nbsp;<INPUTNAME=a8 

type=radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a8  type=radio 
VALUE=4>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a8  type=radio  VALUE=5></FONT></TD> 
</TR> 

<TR> 

<td  width="500"><FONT  FACE=Arial>9.&nbsp;  I  would  accept 
almost  any  type  of  job  assignment  in  order  to  keep  working  for  the  Air 
Force.</FONT> 

</TD> 

<td  width="200"><FONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a9 

type=radio 

VALUE=l>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a9  type=radio 
VALUE=2>&nbsp;&nbsp;&nbsp;&nbsp;<INPUTNAME=a9 

type=radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a9  type=radio 
VALUE=4>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=a9  type=radio  VALUE=5></FONT></TD> 
</TR> 

<TR> 

<td  width="500"><FONT  FACE=Arial>10.&nbsp;  Deciding  to 
join  the  Air  Force  was  a  definite  mistake  on  my  part.  </FONT> 

</TD> 

<td  width="200"xFONT  FACE=Arial>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=alO 

type=radio 

VALUE=l>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=alO  type=radio 
VALUE=2>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=alO 

type=radio  VALUE=3>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=alO  type=radio 
VALUE=4>&nbsp;&nbsp;&nbsp;&nbsp;<INPUT  NAME=alO  type=radio 
VALUE=5x/FONTx/TD> 

</TR> 

</TABLE> 

</CENTER> 

<P  align=left>Do  you  have  any  additional  comments?<BRxCENTER><TEXTAREA  id=comments 
NAME=comments  style="HEIGHT:  75px;  WIDTH:  667px"x/TEXTAREA>  </CENTER> 

<P></P> 

<P  align=left> 

<INPUT  NAME="genderl "  VALUE="<%=gender2%>"  type=hidden> 

<INPUT  NAME="rankl”  VALUE="<%=rank2%>"  type=hidden> 

<INPUT NAME-'afscl"  VALUE="<%=afsc2%>"  type=hidden> 

<INPUT  NAME="majcoml "  VALUE=”<%=majcom2%>"  type=hidden> 

<INPUT  NAME="educationl "  VALUE="<%=education2%>"  type=hidden> 

<INPUT  NAME="marriedl"  VALUE="<%=married2%>"  type=hidden> 

<INPUT  NAME="dependsl "  VALUE="<%=depends2%>"  type=hidden> 

<INPUT  NAME-'cityl”  VALUE="<%=city2%>”  type=hidden> 

<INPUTNAME-'incomel"  VALUE=”<%=income2%>"  type=hidden> 

<INPUT  NAME="yearsl"  VALUE="<%=years2%>"  type=hidden>  </P> 

<DIVx/DIV> 

<P  align=centerxFONT  FACE=ArialxINPUT  TYPE- ’submit"  VALUE="Submit  answers"  id=submit 
NAME=submit> 

</F  ONTx/Px/TDx/TRx/T  ABLE> 
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<DIV></DIV> 

<SCRIPT  LANGU  AGE- ’  VB  SCRIPT,?> 

<!— 

Function  SubmitOnClick 
Dim  my  Form 

Set  myForm  =  document.forml 
Submit_OnClick=True 

If  (Not((myForm.a  1(0). checked)  or  (myForm.al(l).checked)  or  (myForm.al(2).checked)  or 
(myForm.al (3). checked)  or  (myForm.a  1(4). checked)))  Then 
MsgBox  "Please  select  an  option  for  Question  1",0, "Missing  Information" 

SubmitOnClick  =  False 

Elself  (Not((myForm.a2(0). checked)  or  (myForm. a2(l).checked)  or  (myForm.a2(2). checked)  or 
(myForm.a2(3). checked)  or  (myForm.a2(4). checked)))  Then 
MsgBox  "Please  select  an  option  for  Question  2", 0, "Missing  Information" 

Submit_OnClick  =  False 

Elself  (Not((myForm.a3(0). checked)  or  (myForm. a3(l) .checked)  or  (myForm.a3(2).checked)  or 
(myForm.a3(3). checked)  or  (myForm.a3(4).checked)))  Then 
MsgBox  "Please  select  an  option  for  Question  3", 0, "Missing  Information" 

SubmitOnClick  =  False 

Elself  (Not((myForm.a4(0) .checked)  or  (myForm. a4(l) .checked)  or  (myForm.a4(2).checked)  or 
(myForm.a4(3).checked)  or  (myForm.  a4(4)  .checked)))  Then 
MsgBox  "Please  select  an  option  for  Question  4",0J"Missing  Information" 

SubmitOnClick  =  False 

Elself  (Not((myForm.a5(0)xhecked)  or  (myForm.a5(l).checked)  or  (myForm. a5 (2). checked)  or 
(myForm.a5(3).checked)  or  (myForm.a5(4) .checked)))  Then 
MsgBox  "Please  select  an  option  for  Question  5",0, "Missing  Information" 

SubmitOnClick  =  False 

Elself  (Not((myForm.a6(0). checked)  or  (myForm.a6(l ).checked)  or  (myForm.a6(2).checked)  or 
(myForm.a6(3).checked)  or  (myForm.a6(4) .checked)))  Then 
MsgBox  "Please  select  an  option  for  Question  6",0, "Missing  Information" 

Submit_OnClick  =  False 

Elself  (Not((myF orm. a7 (O).checked)  or  (myForm.a7(l ).checked)  or  (myForm.a7(2).checked)  or 
(myForm.a7(3).checked)  or  (myForm.a7(4).checked)))  Then 
MsgBox  "Please  select  an  option  for  Question  7",0,"Missing  Information" 

SubmitOnClick  =  False 

Elself  (Not((myForm.a8(0).checked)  or  (myForm.a8(l  ).checked)  or  (myForm.a8(2).checked)  or 
(myForm.a8(3).checked)  or  (myForm.a8(4) .checked)))  Then 
MsgBox  "Please  select  an  option  for  Question  8", 0, "Missing  Information" 

Submit_OnClick  =  False 

Elself  (Not((myForm.a9(0). checked)  or  (myForm.a9(l) .checked)  or  (myForm.a9(2) .checked)  or 
(myForm.a9(3).checked)  or  (myForm.a9(4).checked)))  Then 
MsgBox  "Please  select  an  option  for  Question  9",0, "Missing  Information" 

SubmitOnClick  =  False 

Elself  (Not((myForm.a  10(0). checked)  or  (myForm.a  10(1) .checked)  or  (myForm. a  10(2) .checked)  or 
(myForm.a  10(3) .checked)  or  (myForm. a  10(4) .checked)))  Then 
MsgBox  "Please  select  an  option  for  Question  10",0, "Missing  Information" 

Submit_OnClick  =  False 
End  If 

End  Function 

— > 

</SCRIPT> 

</FORM> 

<% 
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case  "4" 

ql  =  Request.Form("al") 

q2  =  Request.Form("a2") 

q3  =  Request.  Form("a3") 

q4  =  Request.Form("a4") 

q5  =  Request.Form(Ma5,,) 

q6  -  Request.Form("a6") 

q7  =  Request.Form("a7") 

q8  =  Request.Form("a8") 

q9  =  Request.Form("a9") 

qlO=  Request.Form("alO") 

married2  =  Request.Form(  "married  1") 

depends2  =  Request.Form( "depends  1 M) 

years2  =  Request.  Form(  "years  1") 

city2  =  Request.Form("cityl") 

income2  -  Request.Form("incomer') 

gender2  =  Request.  Form("  gender  1") 

rank2  =  Request.Form(Mrankl") 

afsc2  =  Request.Form(,,afscr') 

majcom2  -  Request.Form("majcoml") 

education2  =  Request.Form("educationr’) 

comments  1  =  Request.  Form(  "comments") 

msgbody  =  ’’Date:  "  &  date  &  chr(13)  &  "Time:  ”  &  time  &  chr(13)  &  "IP:  "  &  ip  &  chr(13)  &  "Gender:  " 
&  gender2  &  chr(13)  &  "Rank:  "  &  rank2  &  chr(13)  &  "AFSC:  "  &  afsc2  &  chr(13)  &  "MAJCOM:  "  & 
majcom2  &  chr(13)  &  "Education:  "  &  education2  &  chr(13)  &  "Married:  "  &  married2  &  chr(13)  & 
"Dependents:  "  &  depends2  &  chr(13)  &  "Years:  "  &  years2  &  chr(13)  &  "City:  "  &  city2  &  chr(13)  & 
"Income:  "  &  income2  &  chr(13)  &  "1.  "  &  ql  &  chr(13)  &  "2.  "  &  q2  &  chr(13)  &  "3.  "  &  q3  &  chr(13)  & 
"4.  "  &  q4  &  chr(13)  &  "5.  "  &  q5  &  chr(13)  &  "6.  "  &  q6  &  chr(13)  &  "7.  "  &  q7  &  chr(13)  &  "8.  "  &  q8 
&  chr(13)  &  "9.  "  &  q9  &  chr(13)  &  "10.  "  &  qlO  &  chr(13)  &  "comments:  "  &  commentsl  &  chr(13) 

'  mail  file 

set  objMail  =  Server.CreateObject("CDONTS.NewMail") 

objMail.From  =  "albert.franke@afit.af.mil" 

objMail. To  =  "albert,  franke@afit.af.mil" 

objMail. Subject  =  "Computer  Survey  Dis- Similar  Results" 

objMail. Body  =  msgbody 

objMail.Send() 

set  objMail  =  nothing 

'  Place  information  into  database 

Set  OBJdbConnection  =  Server.CreateObject("ADODB. Connection") 

OBJdbConnection.Open  "survey" 

string  1="INSERT  INTO  survey2  VALUES  (" 

sql  -  string  1  &  "m  &  date  &  "7"  &  time  &  "7"  &  gender2  &  "7"  &  rank2  &  "7"  &  afsc2  &  "7"  & 
majcom2  &  "7"  &  education  &  "7”  &  married2  &  "7"  &  depends2  &  "7"  &  years2  &  ”7"  &  city2  &  "7" 

&  income2  &  "7”  &  ql  &  "7"  &  q2  &  "7"  &  q3  &  "7"  &  q4  &  "7"  &  q5  &  "7”  &  q6  &  "7"  &  q7  &  "7"  & 

q8  &  "7"  &  q9  &  "7"  &  qlO  &  ”7"  &  ip  &  T 

OBJdbConnection.Execute(sql) 

OBJdbConnection.  close 

%><font  color="#ff0000"  size="4"  ></font> 

<P  align=center><font  color="#ff0000"  size="4"  > 
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</font><FONT  FACE-Arial>&nbsp;</FONT><FONT  color=#ffOOOO  FACE="Wide  Latin" 
size-4><F0NT  color=#ffOOOO  FACE="Wide  Latin"  size=4></F0NT></F0NT></P> 

<P  align=center><F ONT  color=#ff0000  FACE="Wide  Latin"  size=4><FONT  color=#ff0000 
FACE=MWide  Latin"  size=4>Data  sent!  &nbsp;  </FONT></P> 

<P  align=center><FONT  color=#ffD000  FACE- ’Wide  Latin"  size=4>Thank  you  for 
submitting  your  views  on  the  Air  Force.</FONT></P></FONT></FONT></BODY> 
</HTML> 

<% 

end  select 

Function  Strip(var) 
variable=Instr(var,"m) 
if  variable  <>  0  then 
Strip-'  " 
else 

Strip=var 
end  if 

End  Function 

%> 
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1.  I  am  willing  to  put  in  a  great  deal  of  effort  beyond  that  normally  expected  in 
order  to  help  the  Air  Force  be  successful. 


Question  1  Responses 

N 

Mean 

Standard 

Deviation 

Mode 

1  =  Disagree 

2  =  Slightly  Disagree 

3=  Neutral 

4=  Slightly  Agree 

5=  Agree 

Overall:  Paper 

Computer-simple 

Computer-complex 

143 

4.181818 

0.954134 

5 

126 

4.268293 

0.923936 

5 

116 

4.155172 

0.938115 

4 

Men:  Paper 

Computer-simple 

Computer-complex 

67 

4.223881 

0.884563 

5 

53 

4.054407 

1 .002271 

4 

64 

4.296875 

0.885145 

5 

Women:  Paper 

Computer-simple 

Computer-complex 

76 

4.144737 

1.015926 

5 

73 

4.385714 

0.905584 

5 

52 

3.980769 

0.980004 

4 

Officer:  Paper 

Computer-simple 

Computer-complex 

88 

4.318182 

0.878157 

5 

66 

4.421875 

0.831993 

5 

68 

4.117647 

0.832011 

4 

Enlisted:  Paper 

Computer-simple 

Computer-complex 

55 

3.963636 

1 .035725 

4 

60 

4.109375 

0.961434 

4 

48 

4.212766 

1.082191 

5 

2. 1  feel  very  little  loyalty  to  the  Air  Force. 


Question  2  Responses 

N 

Mean 

Standard 

Deviation 

Mode 

1  =  Disagree 

2  =  Slightly  Disagree 

3=  Neutral 

4=  Slightly  Agree 

5=  Agree 

Overall:  Paper 

Computer-simple 

Computer-complex 

143 

2.090909 

1.352666 

i 

126 

2.073171 

1.415203 

i 

116 

2.103448 

1.281163 

i 

Men:  Paper 

Computer-simple 

Computer-complex 

67 

2.044776 

1.319238 

1 

53 

2.116344 

1.339395 

i 

64 

2.125000 

1.290994 

i 

Women:  Paper 

Computer-simple 

Computer-complex 

76 

2.131579 

1.388929 

i 

73 

2.014286 

1.459530 

i 

52 

2.076923 

1.281025 

i 

Officer:  Paper 

Computer-simple 

Computer-complex 

88 

1.931818 

1.362852 

i 

66 

1 .843750 

1.324000 

i 

68 

1 .985294 

1.070610 

i 

Enlisted:  Paper 

Computer-simple 

Computer-complex 

55 

2.345455 

1 .308352 

i 

60 

2.265625 

1.439160 

i 

48 

2.234043 

1.521052 

i 
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3. 1  find  that  my  values  and  those  of  the  Air  Force  are  very  similar. 


Question  3  Responses 

N 

Mean 

Standard 

Deviation 

Mode 

1  =  Disagree 

2  =  Slightly  Disagree 

3=  Neutral 

4=  Slightly  Agree 

5=  Agree 

Overall:  Paper 

Computer-simple 

Computer-complex 

143 

3.86014 

1.091590 

4 

126 

3.869919 

1 .023970 

4 

116 

3.887931 

0.920978 

4 

Men:  Paper 

Computer-simple 

Computer-complex 

67 

3.731343 

1.162313 

4 

53 

3.855549 

1.051493 

4 

64 

3.937500 

0.833333 

4 

Women:  Paper 

Computer-simple 

Computer-complex 

76 

3.973684 

1.019460 

4 

73 

3.842857 

1 .044457 

4 

52 

3.826923 

1 .023664 

4 

Officer:  Paper 

Computer-simple 

Computer-complex 

88 

4.079545 

0.937367 

4 

66 

4.046875 

0.982864 

4 

68 

3.985294 

0.717282 

4 

Enlisted:  Paper 

Computer-simple 

Computer-complex 

55 

3.509091 

1 .230368 

5 

60 

3.703125 

1 .002848 

4 

48 

o 

CO 

CM 

N- 

00 

1.136396 

4 

4. 1  am  proud  to  tell  others  that  I  am  part  of  the  Air  Force. 


Question  4  Responses 

N 

Mean 

Standard 

Deviation 

Mode 

1  =  Disagree 

2  =  Slightly  Disagree 

3=  Neutral 

4=  Slightly  Agree 

5=  Agree 

Overall:  Paper 

Computer-simple 

Computer-complex 

MM 

4.454545 

0.861764 

5 

wm 

4.479675 

0.761250 

5 

116 

4.310345 

0.936355 

5 

Men:  Paper 

Computer-simple 

Computer-complex 

67 

4.462687 

0.784864 

5 

53 

4.307013 

0.942649 

5 

64 

4.390625 

0.847352 

5 

Women:  Paper 

Computer-simple 

Computer-complex 

76 

4.447368 

0.929441 

5 

73 

4.571429 

0.693059 

5 

52 

4.211538 

1 .035385 

5 

Officer:  Paper 

Computer-simple 

Computer-complex 

88 

4.556818 

0.755941 

5 

66 

4.671875 

0.592404 

5 

68 

4.308824 

0.848749 

5 

Enlisted:  Paper 

Computer-simple 

Computer-complex 

55 

4.290909 

0.993921 

5 

60 

4.312500 

0.852168 

5 

48 

4.319149 

1 .065392 

5 
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5. 1  am  extremely  glad  that  I  chose  the  Air  Force  over  other  work  opportunities  I 
was  considering  at  the  time  I  joined. 


Question  5  Responses 

N 

Mean 

Standard 

Deviation 

Mode 

1  =  Disagree 

2  =  Slightly  Disagree 

3=  Neutral 

4=  Slightly  Agree 

5=  Agree 

Overall:  Paper 

Computer-simple 

Computer-complex 

143 

4.027972 

1.150285 

5 

126 

4.130081 

1.115901 

5 

116 

3.982759 

1.172012 

5 

Men:  Paper 

Computer-simple 

Computer-complex 

67 

4.014925 

1.199653 

5 

53 

3.895480 

1.251248 

5 

64 

4.093750 

1 .094267 

5 

Women:  Paper 

Computer-simple 

Computer-complex 

76 

4.039474 

1.112844 

5 

73 

4.2857 1 4' 

1 .009274 

5 

52 

3.846154 

1.258456 

5 

Officer:  Paper 

Computer-simple 

Computer-complex 

88 

4.068182 

1.048211 

5 

66 

4.250000 

1.069045 

5 

68 

5 

Enlisted:  Paper 

55 

5 

Computer-simple 

60 

4.046875 

1.132909 

5 

Computer-complex 

48 

4.085106 

1 .265423 

5 

6.  There  is  little  to  be  gained  by  staying  in  the  Air  Force  until  retirement. 


Question  6  Responses 

N 

Mean 

Standard 

Deviation 

Mode 

Overall:  Paper 

143 

2.559441 

1.417309 

1 

Computer-simple 

126 

2.463415 

1.380738 

1 

Computer-complex 

116 

2.629310 

1.288951 

2 

1  =  Disagree 

Men:  Paper 

67 

2.462687 

1.363244 

2 

Computer-simple 

53 

2.587715 

1 .268856 

2 

2  =  Slightly  Disagree 

Computer-complex 

64 

2.500000 

1.247219 

2 

Women:  Paper 

76 

2.644737 

1.467006 

1 

3=  Neutral 

Computer-simple 

73 

2.342857 

1.443483 

1 

4=  Slightly  Agree 

Computer-complex 

52 

2.788462 

1.333380 

4 

Officer:  Paper 

88 

2.386364 

1.393184 

1 

5=  Agree 

Computer-simple 

66 

2.218750 

1.314978 

1 

Computer-complex 

68 

2.544118 

1.157834 

2 

Enlisted:  Paper 

55 

2.836364 

1.424178 

1 

Computer-simple 

60 

2.671875 

1.392208 

1 

Computer-complex 

48 

2.744681 

1.466556 

1 
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7. 1  really  care  about  the  fate  of  the  Air  Force. 


Question  7  Responses 

N 

Mean 

Standard 

Deviation 

Mode 

1  =  Disagree 

2  =  Slightly  Disagree 

3=  Neutral 

4=  Slightly  Agree 

5=  Agree 

Overall:  Paper 

Computer-simple 

Computer-complex 

143 

4.335664 

0.956402 

5 

126 

4.317073 

0.880732 

5 

116 

4.129310 

0.982722 

5 

Men:  Paper 

Computer-simple 

Computer-complex 

67 

4.373134 

0.950850 

5 

53 

4.108921 

1 .023980 

5 

64 

4.250000 

0.992032 

5 

Women:  Paper 

Computer-simple 

Computer-complex 

76 

4.302632 

0.966364 

5 

73 

4.442857 

0.810005 

5 

52 

3.980769 

0.959787 

4 

Officer:  Paper 

Computer-simple 

Computer-complex 

88 

4.500000 

0.896994 

5 

66 

4.500000 

0.734631 

5 

4.191176 

0.814929 

5 

Enlisted:  Paper 

Computer-simple 

Computer-complex 

4.072727 

0.997303 

5 

4.140625 

0.957298 

5 

ESI 

4.063830 

1.186963 

5 

8.  For  me,  this  is  the  best  of  all  possible  organizations  to  work  for. 


Question  8  Responses 

N 

Mean 

Standard 

Deviation 

Mode 

1  =  Disagree 

2  =  Slightly  Disagree 

3=  Neutral 

4=  Slightly  Agree 

5=  Agree 

Overall:  Paper 

Computer-simple 

Computer-complex 

143 

3.251748 

1.207188 

4 

126 

3.495935 

1.282761 

4 

116 

3.155172 

1 .205808 

4 

Men:  Paper 

Computer-simple 

Computer-complex 

67 

3.208955 

1.174699 

3 

53 

3.062035 

1 .384033 

4 

64 

3.171875 

1 .254259 

4 

Women:  Paper 

Computer-simple 

Computer-complex 

76 

3.289474 

1.241674 

4 

73 

3.814286 

1 .094027 

4 

52 

3.134615 

1.155190 

3 

Officer:  Paper 

Computer-simple 

Computer-complex 

88 

3.295455 

1.146299 

4 

66 

3.312500 

1.307791 

4 

3.102941 

1.182423 

4 

Enlisted:  Paper 

Computer-simple 

Computer-complex 

3.181818 

1 .306549 

3 

3.687500 

1 .206793 

4 

3.276596 

1.210500 

4 
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9.  I  would  accept  almost  any  type  of  job  assignment  in  order  to  keep  working  for 
the  Air  Force. 


Question  9  Responses 

N 

Mean 

Standard 

Deviation 

Mode 

1  =  Disagree 

2  =  Slightly  Disagree 

3=  Neutral 

4=  Slightly  Agree 

5=  Agree 

Overall:  Paper 

Computer-simple 

Computer-complex 

143 

2.307692 

1.234657 

1 

126 

2.609756 

1.364864 

1 

116 

2.336207 

1.291739 

1 

Men:  Paper 

Computer-simple 

Computer-complex 

67 

2.223881 

1.228708 

1 

53 

2.424669 

1 .297548 

1 

64 

2.343750 

1 .359490 

1 

Women:  Paper 

Computer-simple 

Computer-complex 

76 

2.381579 

1 .243298 

1 

73 

2.714286 

1.405402 

1 

52 

1.216249 

1 

Officer:  Paper 

Computer-simple 

Computer-complex 

88 

1 .263382 

1 

66 

HEM 

1.368582 

1 

68 

2.323529 

1 .263799 

1 

Enlisted:  Paper 

Computer-simple 

Computer-complex 

55 

2.181818 

1.187760 

1 

60 

2.500000 

1.356934 

2 

48 

2.382979 

1.344121 

1 

10.  Deciding  to  join  the  Air  Force  was  a  definite  mistake  on  my  part. 


Question  10  Responses 

N 

Mean 

Standard 

Deviation 

Mode 

1  =  Disagree 

2  =  Slightly  Disagree 

3=  Neutral 

4=  Slightly  Agree 

5=  Agree 

Overall:  Paper 

Computer-simple 

Computer-complex 

143 

1.566434 

1.017865 

1 

126 

1.471545 

0.952350 

1 

116 

1.663793 

1.149245 

1 

Men:  Paper 

Computer-simple 

Computer-complex 

67 

1.552239 

1 .004288 

1 

53 

1.528488 

1.023904 

1 

64 

1.562500 

1.021592 

1 

Women:  Paper 

Computer-simple 

Computer-complex 

76 

1.578947 

1.036187 

1 

73 

1.414286 

0.876304 

1 

52 

1.788462 

1.288509 

1 

Officer:  Paper 

Computer-simple 

Computer-complex 

88 

1.511364 

0.982541 

1 

66 

1.453125 

0.941625 

1 

68 

1.705882 

1.154331 

1 

Enlisted:  Paper 

Computer-simple 

Computer-complex 

55 

1 .654545 

1.075281 

1 

60 

1.500000 

0.942809 

1 

48 

1 
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